arXiv CSopen access

Lottery Tickets Are Not Deployment Tickets

Lottery Tickets Are Not Deployment Tickets Bum Jun Kim∗ The University of Tokyo [email protected]

arXiv:2607.27031v1 [cs.LG] 29 Jul 2026

Abstract Reports on how sparsification, compression, and lottery tickets change model behavior have been mixed in the prior literature, with beneficial effects observed in some studies and adverse effects in others. Moreover, prior work has not considered actual deployment conditions, where decision logic is already fixed for the incumbent. To assess these mixed findings from a practical standpoint, we study the productionreplacement question at the deployment level, namely whether an accuracy-matched lottery ticket or another sparse challenger can replace an incumbent dense model without reconfiguring downstream decision logic. We therefore audit a broad, protocol-specific panel of deployment-relevant behaviors spanning calibration, OOD response, class-level reliability, representations, and downstream policy decisions, and summarize clean-accuracy-excluded deviations with a behavioral-compatibility distance. Across experiments on CIFAR-10, CIFAR-100, Imagenette, Flowers-102, and FGVCAircraft with ResNet, WideResNet, ConvNeXt, and Vision Transformer backbones, sparse candidates repeatedly recover dense-reference accuracy yet remain behaviorally different; in several study-band-matched settings, LTs also show lower corruption accuracy. In small-gap settings with fixed-threshold policy diagnostics, lottery-ticket replacement changes 7% to 10% of accept–review decisions. This churn creates precisely the burden that drop-in replacement is meant to avoid: reconfiguring and revalidating downstream decision logic. These findings establish the limits of clean-accuracy certification: Establishing compatibility with a fixed incumbent is distinct from attributing churn uniquely to sparsity or treating every measured deviation as harmful. Our theory explains the routing result: Even exact pointwise top-1 agreement cannot bound fixed-threshold decision changes, and small confidence shifts near the operating boundary can generate first-order routing churn. Clean-accuracy recovery therefore does not certify policy-compatible sparse replacement; the accuracyrecovery check must be followed by an operating-point compatibility audit and signed service-level checks.

Introduction Frankle and Carbin (2019) introduced the lottery ticket (LT) hypothesis, which states that dense neural networks often contain sparse subnetworks that can be trained in isolation to recover comparable test accuracy. Subsequent ∗

Corresponding author

work connected the success of late-rewound tickets to earlytraining stability (Frankle et al. 2020), compared weight and learning-rate rewinding with fine-tuning (Renda, Frankle, and Carbin 2020), and established large-scale sparsity benchmarks (Gale, Elsen, and Hooker 2019). Sparse solutions also transfer across datasets and downstream tasks (Morcos et al. 2019; Iofinova et al. 2022). The promise extends beyond clean accuracy: Pruning can act as a regularizer that improves generalization (Jin et al. 2022), compression can improve outof-distribution (OOD) robustness (Diffenderfer et al. 2021), sparsity can preserve adversarial robustness and shrink the robust generalization gap (Sehwag et al. 2020; Chen et al. 2022a), and calibration-aware recipes improve ticket reliability (Venkatesh et al. 2020; Lei et al. 2023). Taken together, these studies advocate LTs and other sparse models as practically promising replacements. Other studies, however, report that sparsification changes model behavior: Aggregate accuracy can conceal changes in per-example errors, group-level effects, calibration, robustness, and uncertainty under shift (Hooker et al. 2019; Liebenwein et al. 2021; Tran et al. 2022; Dutta et al. 2024; Chen et al. 2022b; Tong et al. 2026). Most directly, Chen et al. (2022b) find that, at appropriate sparsity levels, accuracypreserving LTs can match or outperform their dense counterparts in distribution-shift generalization, uncertainty, interpretability, and loss geometry, although generalization to certain shifts and uncertainty are more sensitive to sparsification. Model-update work likewise shows that aggregate improvement can coexist with backward-incompatible errors or user-facing churn (Bansal et al. 2019; Srivastava et al. 2020). Therefore, evidence about the behavior and practical standing of sparse models is mixed, favorable in some studies and adverse in others. Beyond these mixed reports, prior work also offers little validation under the conditions that an actual deployment imposes. Such a deployment fixes a confidence threshold and a review budget for the incumbent and commits downstream components to the resulting accept–review actions. Existing sparse evaluations hold neither the threshold nor the review budget fixed, leaving a gap that clean-accuracy recovery cannot close. Chen et al. (2022b) recompute each diagnostic separately for the sparse subnetwork, so the evaluation grades an LT as a freshly commissioned model rather than as a replacement inserted into decision logic already fixed for the

(a) Incumbent in production

(b) Drop-in sparse swap

inputs

(c) Same gate, decisions flip

τD

swap in

Dense incumbent

dense 94.6% 94.6%

confidence gate

90%

human review

10%

review

sparse

accept

clean accuracy

τD auto-accept

sparse

dense

τD

7–10%

matched

of accept–review decisions flip

unchanged

downstream logic untouched

!

downstream logic must be revalidated

Figure 1: The production-replacement scenario at the operating-point level. The diagram first shows a deployment that configures a confidence gate for the dense incumbent. Inputs whose confidence falls below the cutoff τD are routed to human review under a fixed review budget, and the rest are auto-accepted by the downstream logic. Next, a sparse challenger that has recovered the incumbent’s clean accuracy is swapped in as a strict drop-in replacement, leaving τD and the inherited downstream logic unchanged. Finally, the same frozen gate still produces different actions because confidence can cross τD even when the top-1 label does not. Crossings occur in both directions, so the review rate can be preserved while individual routing decisions change. AcceptFlipτD reaches 7% to 10% in small-gap settings with fixed-threshold policy diagnostics. incumbent; an LT can therefore pass every such diagnostic and still change what that logic does. Mitra, Schwalbe, and Klein (2024) likewise report calibration and corruption robustness of post-hoc pruned networks as standalone benchmark metrics, but an aggregate calibration score does not reveal whether confidence mass has moved across the particular cutoff a deployment consumes, so unchanged or improved calibration is compatible with substantially changed routing. Tong et al. (2026) restore coverage with a conformal benchmark by calibrating each compressed model. This model-specific recalibration may be appropriate, but the recalibration constitutes a coordinated model-and-policy update rather than a strict drop-in swap: The new rule can require renewed validation of review capacity, accepted risk, monitoring, and application-specific approval. These procedures consume personnel time and operational resources, imposing costs beyond the model swap itself. Our audit asks whether an accuracy-matched challenger can be deployed free from that work. Chee et al. (2022) and Dutta et al. (2024) do compare a compressed model against the corresponding baseline example by example, yet the compared object is the model output rather than the action that inherited downstream logic takes on that output. Neither agreement constraints enforced during compression nor post-compression measurements of answer flips or output distances constrain the routing decision, because an unchanged answer can still cross a fixed confidence gate while a changed answer that both models send to review costs the deployment nothing. Deployment supplies exactly such logic: Production classifiers feed confidence-based selective policies (Geifman and El-Yaniv 2017), OOD rejection and shift monitoring (Hendrycks and Gimpel 2017), robustness checks

(Hendrycks and Dietterich 2019), and monitoring and retraining pipelines (Sculley et al. 2015). Such gates are used in practice: Amazon Rekognition triggers human review for content moderation (Amazon Web Services 2026), and a classifier deployed in the Very Long Baseline Array Fast Radio Transients Experiment reserved low-confidence transient candidates for review (Wagstaff et al. 2016). Because models with similar held-out accuracy can still differ in calibrated confidence, rejected inputs, class-specific errors, and internal representations (Breiman 2001; D’Amour et al. 2022), clean-accuracy recovery leaves these inherited dependencies untested. Figure 1 summarizes the resulting replacement question. In summary, prior work not only offers mixed evidence on the benefits and drawbacks of LTs and other sparse models but also leaves a deployment-validation gap: Existing evaluations do not test sparse challengers against the incumbent operating point for which a running system is already configured. To address this gap, we study whether accuracy-matched LTs and other sparse models can serve as strict drop-in replacements and introduce a method-agnostic operating-point compatibility audit. The challenger must first satisfy a clean-accuracy constraint; the audit then separates two release questions: signed service-level preservation and backward-compatible continuity with the incumbent. The audit pairs each challenger with the corresponding dense reference, holds the downstream decision rule fixed, measures accept–review changes under a dense-derived threshold, and summarizes the remaining active coordinates with a cleanaccuracy-excluded behavioral-compatibility distance. Our central result is that clean-accuracy recovery does not certify compatibility with an incumbent operating point. Em-

pirically, accuracy-matched sparse candidates remain measurably different from the corresponding dense references on active audit coordinates, and in several study-band-matched settings LTs also have lower corruption accuracy. The theory shows that even exact pointwise top-1 agreement cannot bound fixed-threshold decision changes and characterizes how confidence movement near the threshold produces routing flips. Together, the audit, theory, and experiments shift the replacement question from clean-accuracy recovery alone to preservation of the incumbent operating point and relevant signed service levels.

Behavioral Compatibility Audit This section makes the strict drop-in replacement criterion operational in two steps. We first describe each sparse candidate using a clean-accuracy-excluded behavior vector that covers calibration, OOD response, dense-threshold policy decisions, and combined reference and reliability diagnostics. We then impose clean accuracy as a separate matching constraint and measure the standardized distance between the candidate’s behavior and that of the dense reference. The distance supplies a comparative compatibility screen, while signed coordinates supply service-level evidence. Statistical equivalence and application-specific utility are evaluated against separate, predeclared release criteria. Throughout, policy denotes the downstream rule that maps model confidence to an operational action. The dense reference is the matched, trained, unpruned model, and a sparse candidate is a trained sparse alternative evaluated as a potential replacement. We use network and subnetwork when the architectural or connectivity distinction is relevant. We collectively call the inherited downstream dependencies and their acceptable tolerances the replacement contract. The contract’s contents are application specific.

Behavioral Coordinates Coordinate-Selection Principle. We choose coordinates based on the downstream dependency that each coordinate probes, not because every coordinate is required in every deployment. The calibration block audits confidence consumers; the OOD and corruption coordinates audit distribution-shift handling; the policy block audits an inherited accept–review gate; and the reference and reliability block audits class-specific service levels, privacy exposure, per-example output continuity, and, when intermediate features are consumed, representation compatibility. Each quantity can change while clean aggregate accuracy remains fixed. The vector therefore defines a protocol-specific audit panel for common replacement dependencies, with applicationspecific coordinates and costs supplied by the replacement contract. This audit panel contains two logically distinct types of evidence. Signed service-level coordinates, such as corruption accuracy and worst-class accuracy, support directional service-level comparisons. Reference-paired compatibility coordinates, such as prediction or routing disagreement, measure continuity with the incumbent. Clean-Accuracy-Excluded Behavior Vector. Let (X, Y ) denote a clean input–label pair with class set Y :=

{1, . . . , K}, let fD be a dense reference classifier, and let f be a sparse candidate classifier. For any predictor g, let pg (y | x) be the predictive probability assigned by g, ŷg (x) := arg maxy pg (y | x) the prediction made by g under a fixed tie-breaking rule, and cg (x) := maxy pg (y | x) the maximum confidence produced by g. Let τD be the densethreshold policy cutoff defined below. Clean aggregate accuracy, Acc(g) := Pr[ŷg (X) = Y ], is evaluated separately as an explicit matching constraint. Throughout, clean-accuracyexcluded means excluding only this matched clean aggregate accuracy; accuracy-like diagnostics such as worst-class accuracy and corruption accuracy (RobAcc) remain behavioral coordinates. The coordinate set below includes expected calibration error (ECE), negative log-likelihood (NLL), OOD area under the receiver operating characteristic curve (AUROC), in-distribution false-alarm rate at 95% OOD recall (IDFAR95OOD ), membership inference attack (MIA) area under the curve (AUC) excess, and RobAcc. Detailed definitions follow. The clean-accuracy-excluded behavior vector is written as a concatenation of calibration, OOD, dense-threshold policy, and combined reference and reliability coordinates: b−acc (f ; fD , τD ) := concat(bcal , bOOD , bpol , bref ), where repeated coordinate arguments are suppressed and bcal := (ECE, NLL, Brier), bOOD := (AUROCOOD , IDFAR95OOD , Conf OOD ), bpol := (ReviewRateτD , AutoErrτD , OODAcceptτD , AcceptFlipτD ), bref := (WorstClassAcc, MIAex , RepDist, PredDisagree, RobAcc). The complete specification includes coordinate-bycoordinate motivation and exact definitions, together with the distance-fit, normalization, active-coordinate, missingvalue, and scale-fallback rules used below. In particular, PredDisagree measures top-1 disagreement with the dense reference, and RobAcc measures corruption accuracy on a corruption distribution. Incumbent-Threshold Policy Coordinates. We also compute decision-level diagnostics when the required validation and test logits and OOD evaluation data are available. We treat a confidence gate configured for the dense reference as part of the inherited replacement contract. We therefore freeze the dense-derived threshold rather than recalibrating the threshold for each candidate: Candidate-specific recalibration would define a coordinated model-and-policy update rather than the strict drop-in replacement audited here. In our audit, the fixed binary gate auto-accepts an input when the evaluated model’s confidence on that input is at or above τD and otherwise routes the input to review; policy compatibility means preserving the incumbent’s per-input accept–review actions after model replacement under this unchanged gate. Because inputs below the threshold are sent to review, the dense reference sets τD to the empirical 10% quantile of the dense reference’s validation confidences, τD := q0.10 (cfD (Xval )), so that approximately the

lowest-confidence 10% are reviewed. Finite-sample interpolation and ties can make the realized review rate differ from this nominal 10% target. Production policies may use different thresholds and review budgets; the 10% target supplies a common, auditable, and reproducible proxy operating point. A single prespecified operating point is sufficient for our certification question: If clean-accuracy recovery alone were a sufficient drop-in certificate, an accuracy-matched candidate would need to preserve the inherited action at the audited point. Threshold sweeps extend this test to policy robustness across operating points. Applying the same fixed threshold to a candidate gives ReviewRateτD , AutoErrτD , OODAcceptτD , and AcceptFlipτD . With (X, Y ) drawn from the in-distribution test distribution and Z drawn from the OOD distribution, define af (x) := 1{cf (x) ≥ τD } and aD (x) := 1{cfD (x) ≥ τD }, and let Af := {cf (X) ≥ τD }. Then ReviewRateτD (f ) := Pr[cf (X) < τD ], AutoErrτD (f ) := Pr[ŷf (X) ̸= Y | Af ], OODAcceptτD (f ) := Pr[cf (Z) ≥ τD ], AcceptFlipτD (f, fD ) := Pr[af (X) ̸= aD (X)]. The complete policy specification also gives the directional decomposition, missing-value convention, and operational interpretation of these coordinates. AcceptFlipτD is the fraction of in-distribution examples whose acceptance or review decision changes relative to the dense reference.

Behavioral-Compatibility Distance Diagonal Behavioral-Compatibility Distance. Within each protocol group G, the active behavioral coordinates are standardized using fixed group-specific normalization statistics to obtain Zj (f ). Let MG denote the resulting active coordinate set. When |MG | > 0, the diagonal behavioralcompatibility distance is  1/2 X 1 −acc DBE (f, fD ) :=  [Zj (f ) − Zj (fD )]2  . |MG | j∈MG

If |MG | = 0, the distance is undefined and is reported as missing. Applying the same direction multiplier to both the candidate and the dense reference causes the multiplier to cancel from the compatibility distance. The direction is used only in separate utility-signed summaries. Thus, zero distance means equality of the protocol’s imputed activecoordinate vectors. When the same construction is computed from held-out validation estimates of the coordinates used for selection, we denote the resulting validation distance by b −acc,sel (f, fD ). The Appendix specifies the missing-value D BE rule and reports rank- and covariance-aware sensitivity variants of this summary. Compatibility Rather Than Utility. The behavioralcompatibility distance quantifies the magnitude of the measured behavioral difference between a candidate and the deployed dense reference. A coordinate can improve and still indicate changed replacement behavior. Consequently, a

nonzero distance alone does not establish worse deployment performance. Directional degradation claims in this paper instead rely on signed coordinates, most directly when a candidate’s RobAcc is lower than that of the corresponding dense −acc reference. We therefore report DBE together with signed coordinate values in evaluation results, whereas validationb −acc,sel . Hereafter, behavioral distance time selection uses D BE −acc b −acc,sel when validation data is shorthand for DBE or for D BE are explicitly discussed.

Accuracy Cannot Certify Policy Compatibility A sparse candidate can match the dense model’s clean accuracy but still be incompatible with the incumbent policy. This section establishes why: Top-1 outputs do not identify confidence-based behavior, and threshold policies can amplify confidence movement near the policy threshold. Theorem 1 formalizes the resulting non-identifiability even under exact pointwise top-1 agreement. Proposition 1 and Theorem 2 then characterize finite and local threshold-flip behavior, respectively. Theorem 1 (Dense-threshold policy non-identifiability) Let fD be a K-class predictor, K ≥ 2, with measurable predictive probabilities and a measurable tie-broken prediction ŷfD , and fix τD ∈ (1/K, 1). Let (X, Y ) have a joint clean law with a nonatomic input marginal PX , and define AD := {x : cfD (x) ≥ τD }. For every α ∈ [0, 1], there exists a predictive distribution fα such that ŷfα (x) = ŷfD (x) for every x, Acc(fα ) = Acc(fD ), PredDisagree(fα , fD ) = 0, AcceptFlipτD (fα , fD ) = α. This result goes beyond the familiar observation that equal aggregate accuracy can hide different errors: Even exact pointwise top-1 agreement places no nontrivial universal bound on changes to the inherited dense-threshold policy. Nonatomicity serves only to permit arbitrary real-valued rates; under the uniform empirical law on n distinct, independently addressable examples, the same construction realizes every multiple of 1/n. This result is therefore stronger than the replacement condition used in our experiments. The theorem establishes an information limit of the usual LT certificate rather than a typical-case prediction for every LT: The certificate contains no information that rules out such a worst case. In particular, the observed acceptance-flip rates between 7% and 10% are compatible even with zero prediction disagreement: The label can remain fixed while confidence crosses τD . Theorem 1 is a deliberately distribution-free worst-case result. We next quantify threshold flips, first with a finiteperturbation bound and then with the exact local coefficient linking confidence displacement along a specified local predictor path to policy flips. Define the dense threshold margin and the candidate confidence displacement by Dτ := cfD (X) − τD , ∆c := cf (X) − cfD (X). Unless stated otherwise, probabilities and expectations in the next two results are under the clean-input law PX .

Proposition 1 (Finite threshold-flip bound) For pair (f, fD ) and every ε > 0,

every

AcceptFlipτD (f, fD ) = Pr[Dτ ≥ 0 > Dτ + ∆c ] + Pr[Dτ < 0 ≤ Dτ + ∆c ] ≤ Pr[|Dτ | ≤ ε] + Pr[|∆c | > ε]. (1) Consequently, for every p > 0 with E|∆c |p < ∞, the final term admits the moment bound E|∆c |p . εp For the local analysis, let K ≥ 2, let fD be a K-class predictor, and suppose τD ∈ (1/K, 1). Consider any local path {ft : t > 0} of predictive distributions with a measurable pre-clipping confidence direction V = V (X) such that AcceptFlipτD (f, fD ) ≤ Pr[|Dτ | ≤ ε] +

cft (X) = Π[1/K,1] (cfD (X) + tV ), where Π[1/K,1] denotes clipping to the feasible maximumconfidence range. Because τD is interior to that range, clipping does not alter the acceptance decision. For every z ∈ R, Π[1/K,1] (z) ≥ τD ⇐⇒ z ≥ τD . Theorem 2 (Threshold-boundary amplification) For the local predictor path above, suppose (Dτ , V ) has a joint density with a version h(d, v) such that, for almost every v, d 7→ h(d, v) is continuous at zero. Assume V is essentially bounded and that there R exist d0 > 0 and a measurable h̄ : R → [0, ∞) such that R |v|h̄(v)dv < ∞ and, for almost every v, h(d, v) ≤ h̄(v) for every |d| ≤ d0 . Then Z AcceptFlipτD (ft , fD ) = κτD := |v|h(0, v)dv. lim t t→0+ R (2) Equivalently, Eq. 2 gives the right-sided expansion AcceptFlipτD (ft , fD ) = tκτD + o(t), which has a nonzero first-order term exactly when κτD > 0. We call κτD the threshold-amplification coefficient. The relevant boundary is the confidence level set rather than the top-1 class boundary. These results motivate the reported AcceptFlipτD coordinate and the behavior-aware selection specified in the experimental setup below. The Appendix also gives an extended operational interpretation of all three results.

Experiments Experimental Setup We conduct the primary evaluation on the 10-class Canadian Institute for Advanced Research dataset (CIFAR-10) with an 18-layer residual network (ResNet-18) (Krizhevsky, Hinton et al. 2009; He et al. 2016) at 50%, 80%, 90%, and 95% sparsity. Replications cover the 100-class variant, CIFAR-100, with ResNet-18, and CIFAR-10 with ResNet-34 and a 28layer wide residual network (WideResNet-28-2) (Zagoruyko and Komodakis 2016). For pretrained evaluations, we use

Imagenette (Deng et al. 2009) with ConvNeXt-Tiny (Liu et al. 2022) and the Tiny variant of the Vision Transformer (ViT), denoted ViT-Tiny (Dosovitskiy et al. 2021), followed by Flowers-102 (Nilsback and Zisserman 2008) and FineGrained Visual Classification of Aircraft (FGVC-Aircraft) (Maji et al. 2013) transfer evaluations. We evaluate accuracy-only selection and the BehaviorPreserving Lottery Ticket (BP-LT) procedure over each eligible sparse pool. The pools and reported baselines include LT, LT reinitialization, which tables abbreviate as LT reinit, global and layerwise random sparse controls, and one-shot and staged magnitude pruning. We abbreviate magnitude pruning as MP and use One-shot MP and Staged MP in tables. Protocol-specific pools additionally include iterative magnitude pruning (IMP), Single-Shot Network Pruning (SNIP), and the Rigged Lottery (RigL). LT Construction. For the reported LT baseline, we first train the matched dense reference to completion, rank all maskable weights globally by their final absolute values, and retain the exact top-k entries required by the target sparsity. This ranking procedure produces a one-shot global magnitude mask. We then instantiate the sparse candidate at the protocol-specific rewinding point and train the surviving weights with the mask enforced throughout optimization. In from-scratch protocols, the rewinding point is the original initialization of the matched dense reference; in pretrained protocols, the rewinding point is the pretrained backbone together with the classifier head’s original random initialization. LT reinitialization uses the same mask but a separately sampled initialization. Dense seed, mask seed, candidate training seed, and candidate identifier are stored separately in the result metadata. For sparsity s, let Fs := {fi }ni=1 denote the fixed eligible pool for one matched dense-reference seed, sparsity, and protocol cell, from which BP-LT selects one existing model. All candidates are selected using held-out validation metrics, and final metrics are recomputed on the test split. Write sel sel d d (fD )|/ϵacc for the normal(fi ) − Acc ∆sel := |Acc i

clean

clean

sel

d clean deized absolute validation accuracy gap, where Acc notes validation clean accuracy and ϵacc > 0 is the accuracyband tolerance, so the validation accuracy band comprises the candidates with ∆sel i ≤ 1. BP-LT. BP-LT selects the in-band candidate minimizing validation behavioral-compatibility distance plus a small normalized accuracy-gap penalty with weight λ: When the validation accuracy band is nonempty, the selected candidate is b −acc,sel (fi , fD ) + λ(∆sel )2 ]. fBP ∈ argmin[D i BE fi ∈Fs : ∆sel i ≤1

Inside the band, behavioral closeness is the primary term, while the quadratic accuracy penalty discourages choosing a candidate near the edge of the allowed band. The penalty is zero at an exact validation-accuracy match and equals λ at the band boundary. This penalty structure implements a soft trade-off within a hard eligibility screen. If the band is empty,

the fallback reverses the emphasis by making the normalized accuracy gap the primary term and squared behavioral distance the secondary term. The fallback uses deterministic tie-breaking and excludes nonfinite validation values. For all reported selections, we set ϵacc = 0.015 and λ = 0.2. We stratify protocol–method groups by recovery quality: A group is study-band-matched when the group’s dense reference passes the dataset-specific accuracy floor and every evaluated test accuracy gap is finite and lies within the prespecified test recovery band of ±1.5 percentage points (pp), near-recovery when the group fails that rule with an absolute mean gap of at most 3 pp, and recovery-stress otherwise. Raw gaps remain reported throughout, so narrower application-specific margins can be applied. The standard CIFAR protocols use Street View House Numbers (SVHN) (Netzer et al. 2011) for OOD evaluation and the CIFAR-10 corruption benchmark (CIFAR-10-C) when available. The Appendix provides the full experimental protocol and reproducibility details, additional results and analyses, complete audit-coordinate and distance specifications, proofs, BP-LT eligibility, fallback, and implementation details, and further discussion, including limitations.

CIFAR-10 ResNet-18 Results Multi-Sparsity Evidence. Table 1 reports the primary CIFAR-10 ResNet-18 multi-sparsity evaluation. The dense reference reaches (94.62 ± 0.26)% accuracy. At 50% sparsity, the LT baseline matches the dense reference’s mean accuracy within 0.01 pp yet has a mean behavioral distance −acc of DBE = 0.648 from that reference. On the protocolstandardized scale, this distance is a root-mean-square separation of about 0.65 units across the active behavioral coordinates. BP-LT keeps the mean accuracy gap near zero and reduces the distance to 0.446. At 80% sparsity, the LT baseline remains close in accuracy and has a distance of 0.501. Accuracy-only selection is much farther away with a distance of 0.890. BP-LT reduces the distance to 0.437 with a 0.35 pp mean accuracy gap. At 90% and 95%, all sparse methods become more strained, but BP-LT still has lower behavioral distance than the LT baseline and accuracy-only selection on average. The signed RobAcc result at 80% is stronger than symmetric incompatibility alone: Although the LT is only 0.16 pp below the dense reference in clean accuracy, the LT’s RobAcc falls from 65.71% to 64.14%, a 1.57 pp loss on corrupted inputs. Table 1 shows that LT baselines are strong in clean accuracy at the 50% and 80% sparsity levels, yet matching clean accuracy does not establish behavioral compatibility: The gap is visible in OOD response, corruption accuracy, representation distance, and policy behavior. The accuracy constraint also matters. Candidates with global random sparse masks can sometimes have a moderate behavioralcompatibility distance and, in some protocol groups, satisfy the test recovery band, while harder or higher-sparsity groups −acc miss the band. We therefore report DBE together with the explicit accuracy-matching status. Larger Sparse-Baseline Set at 80% Sparsity. Appendix Table 6 reports the complementary 50% large-pool compar-

Acc (%)

∆Acc (pp)

−acc DBE

RobAcc (%)

Sparsity (%)

Method

Active params. (millions)

0

Dense

11.17

94.62 ± 0.26

0.00

0.000

65.71

50

LT Accuracy-only BP-LT Random sparse

5.59

94.63 ± 0.10 94.52 ± 0.19 94.62 ± 0.20 94.31 ± 0.15

+0.01 −0.10 0.00 −0.31

0.648 ± 0.555 0.654 ± 0.467 0.446 ± 0.304 0.727 ± 0.404

65.87 65.37 65.13 65.34

80

LT Accuracy-only BP-LT Random sparse

2.24

94.46 ± 0.17 94.38 ± 0.22 94.27 ± 0.51 93.56 ± 0.16

−0.16 −0.24 −0.35 −1.06

0.501 ± 0.248 0.890 ± 0.598 0.437 ± 0.168 0.981 ± 0.434

64.14 64.10 63.77 62.95

90

LT Accuracy-only BP-LT Random sparse

1.13

94.13 ± 0.06 94.06 ± 0.19 93.85 ± 0.20 92.41 ± 0.23

−0.49 −0.56 −0.77 −2.21

1.364 ± 0.455 1.295 ± 0.415 0.995 ± 0.156 1.521 ± 0.359

62.94 63.71 63.85 61.52

95

LT Accuracy-only BP-LT Random sparse

0.57

93.38 ± 0.10 93.50 ± 0.15 93.28 ± 0.20 90.98 ± 0.22

−1.24 −1.12 −1.34 −3.64

1.059 ± 0.095 1.099 ± 0.071 1.014 ± 0.138 2.183 ± 0.211

61.52 61.48 62.15 59.05

Table 1: Multi-sparsity results for CIFAR-10 ResNet-18. Acc is the mean ± standard deviation across evaluated models, and active parameter counts are in millions. ∆Acc is candidate test accuracy minus the matched dense reference’s test accuracy (pp), and RobAcc is corruption accuracy under the setting-specific protocol. This table uses CIFAR-10-C. −acc DBE quantifies the magnitude of the measured behavioral difference from the dense reference, and smaller values indicate closer behavior. These conventions apply to subsequent tables.

ison, and Appendix Table 7 reports the companion 80% protocol with RigL and Staged MP. The 50% comparison shows that behavioral gaps persist with a stronger sparse-baseline set. Because Table 1 already suggests that 80% sparsity is the stricter sparse-recovery stress point, we evaluate the broader CIFAR-10 ResNet-18 comparison there. Table 2 is the largest sparse-baseline comparison at 80% sparsity in this paper. The dense reference’s accuracy remains high at 94.53%. The LT baseline, LT reinitialization, IMP, SNIP, global random sparse, layerwise random sparse, and one-shot MP protocol–method groups all lie within the ±1.5 pp test recovery band and are thus study-band-matched. Yet all of these methods show nonzero measured devia−acc tions from the dense reference. The LT baseline has DBE = 0.784, 4.8% prediction disagreement, and AcceptFlipτD = 8.5%. IMP and SNIP recover accuracy but show similar behavioral distances. Candidates with global random sparse masks are weaker in accuracy and farther in behavior. Layerwise random sparse controls and one-shot MP controls have smaller distances but still change measured coordinates relative to the dense reference. BP-LT selection reduces the behavioral distance from 0.784 for the LT baseline to 0.701 while preserving the accuracy constraint. BP-LT selection does not reduce the headline policy coordinate in this cell: The mean acceptance-flip rate is 8.54% for BP-LT and 8.47% for LT. This outcome is consistent with BP-LT’s role as a multi-coordinate screening rule, not a policy-restoration objective. The Staged MP protocol–method group represents the recovery-stress stratum of the same setting. Under this staged recipe with a 200-epoch sparse training budget, the Staged MP baseline reaches 91.48%, about 3.05 pp below the dense

Method

Acc (%)

∆Acc (pp)

−acc DBE

OOD AUROC

Flip (%)

RobAcc (%)

Policy-Level Shift 12 11

94.53 ± 0.21

0.00

0.000

0.8609

0.00

72.42

94.39 ± 0.05 94.33 ± 0.17 94.25 ± 0.07 94.31 ± 0.09 93.70 ± 0.15 94.34 ± 0.02 94.23 ± 0.16

−0.14 −0.20 −0.29 −0.22 −0.84 −0.19 −0.31

0.784 0.784 0.758 0.800 1.047 0.724 0.655

0.8427 0.8490 0.8520 0.8554 0.8324 0.8487 0.8596

8.47 8.87 9.02 8.87 9.98 8.58 6.92

71.27 71.18 71.40 70.84 70.58 70.79 71.86

Staged MP

91.48 ± 0.21

−3.05

1.940

0.8494

15.23

67.28

6

Accuracy-only BP-LT

94.30 ± 0.09 94.44 ± 0.09

−0.24 −0.09

0.782 0.701

0.8447 0.8511

8.60 8.54

71.40 71.18

5 −1.50

Small dense

88.10 ± 0.20

−6.44

2.995

0.7983

21.59

61.01

Table 2: Large-pool CIFAR-10 ResNet-18 at 80% sparsity. Flip denotes AcceptFlipτD . reference. A companion protocol with RigL and Staged MP adopts a lower post-pruning fine-tuning learning rate; results from this separate protocol group support the same replacement audit conclusion that accuracy recovery alone does not certify continuity.

Replications and Transfer The pattern of accuracy recovery with behavioral gaps recurs across datasets, architectures, and pretrained backbones. Appendix Tables 8, 9, and 10 report the full results. On CIFAR-100 ResNet-18 at 50% sparsity, the LT baseline reaches 74.66%, compared with 75.30% for the dense ref−acc erence, with DBE = 0.768 and a 1.40 pp RobAcc loss. CIFAR-10 ResNet-34 at 80% is within 0.07 pp of the cor−acc responding dense reference but has DBE = 1.029, and CIFAR-10 WideResNet-28-2 at 80% stays inside the test −acc recovery band with a 1.17 pp gap, DBE = 1.149, and a 3.54 pp RobAcc loss. The pretrained Imagenette protocols at 50% sparsity extend the audit to modern backbones. The ConvNeXt-Tiny LT baseline reaches 97.94%, exceeding the corresponding dense reference’s 97.49% clean accu−acc racy, yet still has DBE = 1.205 and flips 9.69% of densethreshold decisions. ViT-Tiny recovers to within 0.28 pp with −acc DBE = 0.422 and AcceptFlipτD = 8.08%. Fine-grained transfer with pretrained ConvNeXt-Tiny repeats the pattern: Flowers-102 at 50% sparsity recovers to within 0.27 pp with −acc DBE = 0.739 and AcceptFlipτD = 7.28%, and FGVCAircraft at 50% sparsity recovers to within 0.93 pp with −acc DBE = 0.658 and AcceptFlipτD = 7.91%. BP-LT lowers the behavioral distance in every one of these settings.

Policy-Level Shifts For protocols with fixed-threshold policy diagnostics, Figure 2 shows that LT baselines in evaluations with |∆Acc| ≤ 3 pp flip roughly 7% to 10% of dense-threshold decisions; grouped method means span 5% to 12%. All plotted grouped method means have accuracy gaps within ±1.5 pp, yet candidates with near-zero gaps and those that exceed dense accuracy flip similarly. Accuracy gap therefore does not predict routing churn, consistent with Theorem 1, Proposition 1, and Theorem 2. Net accept-to-review crossings can raise review arrivals and latency; review-to-accept crossings bypass in-

AcceptFlipτD (%)

Dense LT LT reinit IMP SNIP Random sparse Layerwise random One-shot MP

10 9 8 7

−1.25

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

Accuracy gap from dense reference (points) LT Acc-only

BP-LT MP

IMP

SNIP

RigL

Figure 2: Policy-level shift for small-gap evaluations satisfying |∆Acc| ≤ 3 pp. AcceptFlipτD is the percentage of test inputs receiving opposite accept–review decisions under the fixed dense threshold. This coordinate pools both directions and is not a top-1 label-flip rate. Markers show grouped method means with offsets for overlaps. Area increases with −acc DBE .

cumbent escalation and can alter accepted risk. Offset flows can preserve volume while changing reviewed cases. Acceptance flips thus establish routing non-preservation, not realized cost or harm; quantifying the latter requires directional rates and deployment costs. The Appendix gives full proofs and further interpretation and causal scope.

Conclusion Our deployment-perspective evaluation concludes that matching an incumbent dense model’s clean accuracy does not ensure that an LT is a policy-compatible replacement at the incumbent’s fixed operating point. We show theoretically that the insufficiency of an accuracy-only certificate is structural: Exact top-1 agreement does not identify the densethreshold policy, and near-threshold confidence movement along regular local paths produces policy flips at a first-order rate governed by the threshold-amplification coefficient. Empirically, across a broad range of settings, sparse candidates that recover the accuracy of the corresponding dense references remain measurably different on active audit coordinates, and the sparse candidates’ accept–review changes do not vanish as the accuracy gap approaches zero. Recurring corruption-accuracy losses in study-band-matched settings further show that clean recovery can mask lower deploymentrelevant performance, a risk a signed service-level audit can detect. Behavior-aware selection often reduces aggregate compatibility gaps while preserving the accuracy constraint; policy restoration and individual-coordinate requirements remain separate release checks. Strict drop-in claims therefore need paired compatibility checks and signed service-level requirements alongside clean accuracy. For each dense–sparse comparison, we set the threshold from the dense reference’s validation confidences to target a 10% review rate and apply that threshold unchanged to the sparse candidate.

References Amazon Web Services. 2026. Reviewing Inappropriate Content with Amazon Augmented AI. Amazon Rekognition Developer Guide, https://docs.aws.amazon.com/rekognition/ latest/dg/a2i-rekognition.html. Bahri, D.; and Jiang, H. 2021. Locally Adaptive Label Smoothing Improves Predictive Churn. In ICML, volume 139, 532–542. Bansal, G.; Nushi, B.; Kamar, E.; Weld, D. S.; Lasecki, W. S.; and Horvitz, E. 2019. Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff. In AAAI, 2429–2437. Blalock, D. W.; Ortiz, J. J. G.; Frankle, J.; and Guttag, J. V. 2020. What is the State of Neural Network Pruning? In MLSys. Breiman, L. 2001. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science, 16(3): 199–231. Carlini, N.; Chien, S.; Nasr, M.; Song, S.; Terzis, A.; and Tramèr, F. 2022. Membership Inference Attacks From First Principles. In SP, 1897–1914. Chee, J.; Flynn, M.; Damle, A.; and Sa, C. D. 2022. Model Preserving Compression for Neural Networks. In NeurIPS. Chen, T.; Frankle, J.; Chang, S.; Liu, S.; Zhang, Y.; Carbin, M.; and Wang, Z. 2021. The Lottery Tickets Hypothesis for Supervised and Self-Supervised Pre-Training in Computer Vision Models. In CVPR, 16306–16316. Chen, T.; Zhang, Z.; Wang, P.; Balachandra, S.; Ma, H.; Wang, Z.; and Wang, Z. 2022a. Sparsity Winning Twice: Better Robust Generalization from More Efficient Training. In ICLR. Chen, T.; Zhang, Z.; Wu, J.; Huang, R.; Liu, S.; Chang, S.; and Wang, Z. 2022b. Can You Win Everything with A Lottery Ticket? Trans. Mach. Learn. Res. Coates, A.; Ng, A. Y.; and Lee, H. 2011. An Analysis of Single-Layer Networks in Unsupervised Feature Learning. In AISTATS, volume 15, 215–223. D’Amour, A.; Heller, K. A.; Moldovan, D.; Adlam, B.; Alipanahi, B.; Beutel, A.; Chen, C.; Deaton, J.; Eisenstein, J.; Hoffman, M. D.; Hormozdiari, F.; Houlsby, N.; Hou, S.; Jerfel, G.; Karthikesalingam, A.; Lucic, M.; Ma, Y.; McLean, C. Y.; Mincu, D.; Mitani, A.; Montanari, A.; Nado, Z.; Natarajan, V.; Nielson, C.; Osborne, T. F.; Raman, R.; Ramasamy, K.; Sayres, R.; Schrouff, J.; Seneviratne, M.; Sequeira, S.; Suresh, H.; Veitch, V.; Vladymyrov, M.; Wang, X.; Webster, K.; Yadlowsky, S.; Yun, T.; Zhai, X.; and Sculley, D. 2022. Underspecification Presents Challenges for Credibility in Modern Machine Learning. J. Mach. Learn. Res., 23: 226:1–226:61. Deng, J.; Dong, W.; Socher, R.; Li, L.; Li, K.; and Fei-Fei, L. 2009. ImageNet: A large-scale hierarchical image database. In CVPR, 248–255. Diffenderfer, J.; Bartoldson, B. R.; Chaganti, S.; Zhang, J.; and Kailkhura, B. 2021. A Winning Hand: Compressing Deep Networks Can Improve Out-of-Distribution Robustness. In NeurIPS, 664–676.

Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR. Dutta, A.; Krishnan, S.; Kwatra, N.; and Ramjee, R. 2024. Accuracy is Not All You Need. In NeurIPS. Evci, U.; Gale, T.; Menick, J.; Castro, P. S.; and Elsen, E. 2020. Rigging the Lottery: Making All Tickets Winners. In ICML, volume 119, 2943–2952. Fard, M. M.; Cormier, Q.; Canini, K. R.; and Gupta, M. R. 2016. Launch and Iterate: Reducing Prediction Churn. In NIPS, 3171–3179. Frankle, J.; and Carbin, M. 2019. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. In ICLR. Frankle, J.; Dziugaite, G. K.; Roy, D. M.; and Carbin, M. 2019. The Lottery Ticket Hypothesis at Scale. CoRR, abs/1903.01611. Frankle, J.; Dziugaite, G. K.; Roy, D. M.; and Carbin, M. 2020. Linear Mode Connectivity and the Lottery Ticket Hypothesis. In ICML, volume 119, 3259–3269. Frankle, J.; Dziugaite, G. K.; Roy, D. M.; and Carbin, M. 2021. Pruning Neural Networks at Initialization: Why Are We Missing the Mark? In ICLR. Gale, T.; Elsen, E.; and Hooker, S. 2019. The State of Sparsity in Deep Neural Networks. CoRR, abs/1902.09574. Geifman, Y.; and El-Yaniv, R. 2017. Selective Classification for Deep Neural Networks. In NIPS, 4878–4887. Glenn, W. B.; et al. 1950. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1): 1–3. Guo, C.; Pleiss, G.; Sun, Y.; and Weinberger, K. Q. 2017. On Calibration of Modern Neural Networks. In ICML, volume 70, 1321–1330. Han, S.; Mao, H.; and Dally, W. J. 2016. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding. In ICLR. Han, S.; Pool, J.; Tran, J.; and Dally, W. J. 2015. Learning both Weights and Connections for Efficient Neural Network. In NIPS, 1135–1143. He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep Residual Learning for Image Recognition. In CVPR, 770–778. Hendrycks, D.; and Dietterich, T. G. 2019. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. In ICLR. Hendrycks, D.; and Gimpel, K. 2017. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. In ICLR. Hooker, S.; Courville, A. C.; Dauphin, Y. N.; and Frome, A. 2019. Selective Brain Damage: Measuring the Disparate Impact of Model Pruning. CoRR, abs/1911.05248. Iofinova, E.; Peste, A.; Kurtz, M.; and Alistarh, D. 2022. How Well Do Sparse ImageNet Models Transfer? In CVPR, 12256–12266.

Jin, T.; Carbin, M.; Roy, D. M.; Frankle, J.; and Dziugaite, G. K. 2022. Pruning’s Effect on Generalization Through the Lens of Training and Regularization. In NeurIPS. Kornblith, S.; Norouzi, M.; Lee, H.; and Hinton, G. E. 2019. Similarity of Neural Network Representations Revisited. In ICML, volume 97, 3519–3529. Krizhevsky, A.; Hinton, G.; et al. 2009. Learning multiple layers of features from tiny images. LeCun, Y.; Denker, J. S.; and Solla, S. A. 1989. Optimal Brain Damage. In NIPS, 598–605. Ledoit, O.; and Wolf, M. 2004. A well-conditioned estimator for large-dimensional covariance matrices. Journal of multivariate analysis, 88(2): 365–411. Lee, N.; Ajanthan, T.; and Torr, P. H. S. 2019. Snip: singleShot Network Pruning based on Connection sensitivity. In ICLR. Lei, B.; Zhang, R.; Xu, D.; and Mallick, B. K. 2023. Calibrating the Rigged Lottery: Making All Tickets Reliable. In ICLR. Liebenwein, L.; Baykal, C.; Carter, B.; Gifford, D.; and Rus, D. 2021. Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy. In MLSys. Liu, Z.; Mao, H.; Wu, C.; Feichtenhofer, C.; Darrell, T.; and Xie, S. 2022. A ConvNet for the 2020s. In CVPR, 11966– 11976. Liu, Z.; Sun, M.; Zhou, T.; Huang, G.; and Darrell, T. 2019. Rethinking the Value of Network Pruning. In ICLR. Maji, S.; Rahtu, E.; Kannala, J.; Blaschko, M. B.; and Vedaldi, A. 2013. Fine-Grained Visual Classification of Aircraft. CoRR, abs/1306.5151. Marx, C. T.; Calmon, F. P.; and Ustun, B. 2020. Predictive Multiplicity in Classification. In ICML, volume 119, 6765– 6774. Mitra, P.; Schwalbe, G.; and Klein, N. 2024. Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Study. In CVPR Workshops, 3542–3552. Morcos, A. S.; Yu, H.; Paganini, M.; and Tian, Y. 2019. One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers. In NeurIPS, 4933–4943. Naeini, M. P.; Cooper, G. F.; and Hauskrecht, M. 2015. Obtaining Well Calibrated Probabilities Using Bayesian Binning. In AAAI, 2901–2907. Netzer, Y.; Wang, T.; Coates, A.; Bissacco, A.; Wu, B.; Ng, A. Y.; et al. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, 4. Nilsback, M.; and Zisserman, A. 2008. Automated Flower Classification over a Large Number of Classes. In ICVGIP, 722–729. Ötles, E.; Denton, B. T.; and Wiens, J. 2023. Updating Clinical Risk Stratification Models Using Rank-Based Compatibility: Approaches for Evaluating and Optimizing ClinicianModel Team Performance. In MLHC, volume 219, 529–547.

Renda, A.; Frankle, J.; and Carbin, M. 2020. Comparing Rewinding and Fine-tuning in Neural Network Pruning. In ICLR. Sculley, D.; Holt, G.; Golovin, D.; Davydov, E.; Phillips, T.; Ebner, D.; Chaudhary, V.; Young, M.; Crespo, J.; and Dennison, D. 2015. Hidden Technical Debt in Machine Learning Systems. In NIPS, 2503–2511. Sehwag, V.; Wang, S.; Mittal, P.; and Jana, S. 2020. HYDRA: Pruning Adversarially Robust Neural Networks. In NeurIPS. Shokri, R.; Stronati, M.; Song, C.; and Shmatikov, V. 2017. Membership Inference Attacks Against Machine Learning Models. In IEEE Symposium on Security and Privacy, 3–18. Snoek, J.; Ovadia, Y.; Fertig, E.; Lakshminarayanan, B.; Nowozin, S.; Sculley, D.; Dillon, J. V.; Ren, J.; and Nado, Z. 2019. Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. In NeurIPS, 13969–13980. Srivastava, M.; Nushi, B.; Kamar, E.; Shah, S.; and Horvitz, E. 2020. An Empirical Analysis of Backward Compatibility in Machine Learning Systems. In KDD, 3272–3280. Tanaka, H.; Kunin, D.; Yamins, D. L. K.; and Ganguli, S. 2020. Pruning neural networks without any data by iteratively conserving synaptic flow. In NeurIPS. Tong, Y.; Wang, Y.; Wan, Y.; Zhang, T.; Dong, J.; and Yuan, J. 2026. Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction. CoRR, abs/2606.01850. Tran, C.; Fioretto, F.; Kim, J.; and Naidu, R. 2022. Pruning has a disparate impact on model accuracy. In NeurIPS. Venkatesh, B.; Thiagarajan, J. J.; Thopalli, K.; and Sattigeri, P. 2020. Calibrate and Prune: Improving Reliability of Lottery Tickets Through Prediction Calibration. CoRR, abs/2002.03875. Wagstaff, K. L.; Tang, B.; Thompson, D. R.; Khudikyan, S.; Wyngaard, J.; Deller, A. T.; Palaniswamy, D.; Tingay, S. J.; and Wayth, R. B. 2016. A machine learning classifier for fast radio burst detection at the VLBA. Publications of the Astronomical Society of the Pacific, 128(966): 084503. Wang, C.; Zhang, G.; and Grosse, R. B. 2020. Picking Winning Tickets Before Training by Preserving Gradient Flow. In ICLR. Wightman, R. 2019. PyTorch Image Models. https://github. com/rwightman/pytorch-image-models. Yan, S.; Xiong, Y.; Kundu, K.; Yang, S.; Deng, S.; Wang, M.; Xia, W.; and Soatto, S. 2021. Positive-Congruent Training: Towards Regression-Free Model Updates. In CVPR, 14299– 14308. Yeom, S.; Giacomelli, I.; Fredrikson, M.; and Jha, S. 2018. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. In CSF, 268–282. Zagoruyko, S.; and Komodakis, N. 2016. Wide Residual Networks. In BMVC. Zhu, M.; and Gupta, S. 2018. To Prune, or Not to Prune: Exploring the Efficacy of Pruning for Model Compression. In ICLR (Workshop).

Appendix Sparse-Replacement Test Overview Accuracy Slab

Policy Half-Space

Behavioral Distance

ϵacc τD |∆Accval | ≤ ϵacc LT

−acc DBE

−acc DBE

review

BP-LT

−acc DBE

accept

fD

flip

fD f

BP-LT fD ∆Accval

c(x)

Figure 3: Illustration of the sparse-replacement audit. Sparse candidates first pass through a validation clean-accuracy slab sel sel d clean (fD ). Inside that slab, replacement is screened by d clean (f ) − Acc around the dense reference, where ∆Accval := Acc −acc clean-accuracy-excluded behavioral-compatibility distance (DBE ) to the dense reference’s behavior vector, and BP-LT selects a closer in-band candidate. The final audit measures changes across the acceptance and review boundary after reusing the dense threshold τD . Behavioral Audit Construction Behavioral Coordinates and Distance Details List of Notation Symbol

Meaning

fD ; f, fi s, Fs ; (X, Y ), Z

Dense reference; sparse candidates Sparsity and candidate pool; clean input–label pair and OOD input Predictive distribution; maximum confidence; predicted class Dense confidence threshold; acceptance indicator and event

pf (y | x); cf (x); ŷf (x) τD ; af (x), Af b−acc ; bcal , bOOD , bpol , bref

AcceptFlipτ ; PredDisagree; D RobAcc G, FG , MG xj , x ej , Zj ; dj , µj,G , σj,G −acc b −acc,sel DBE ; DBE

Dτ , ∆c V , h, κτD

d sel , ϵacc ; ∆sel , λ Acc clean i fb fBP , fBP ; ∆Accval , ∆Acc

Drank , DMah

Behavior vector and its calibration, OOD, policy, and combined reference and reliability blocks Acceptance-policy disagreement; top-1 disagreement; corruption accuracy Protocol group; distance-fit rows; active coordinate set Raw, imputed, and standardized coordinate; direction, mean, and scale Test-split and validation-split behavioral-compatibility distances Dense threshold margin and candidate confidence displacement Local confidence direction, joint density, and threshold-amplification coefficient Validation accuracy and tolerance; normalized gap and penalty weight BP-LT choices; signed validation and test accuracy gaps Sensitivity distances

Table 3: Principal notation. Auxiliary symbols and individual behavioral coordinates are defined where introduced.

Coordinate Roles and Calibration Details. The coordinates within a block expose complementary failure modes. ECE measures empirical confidence–accuracy mismatch, while NLL and Brier are proper scoring rules that assess the predictive distribution and penalize confident errors. OOD AUROC measures ranking quality, IDFAR95OOD measures in-distribution false-alarm load at a fixed OOD-recall target, and OOD confidence measures residual confidence on shifted inputs. The policy coordinates translate confidence movement into review volume, accepted error, OOD acceptance, and paired routing changes. Worst-class accuracy detects a class-specific service-level loss hidden by the aggregate. An MIA probes black-box membership exposure, prediction disagreement measures output churn, and corruption accuracy probes performance under the evaluated input shifts. Representation distance is included only as a diagnostic of intermediate-feature compatibility and does not independently establish harm. We compute ECE with 15 equal-width confidence bins, and Brier(f ) is the multiclass Brier score (Naeini, Cooper, and Hauskrecht 2015; Glenn et al. 1950), K X Brier(f ) := E[ (pf (y | X) − 1{Y = y})2 ]. y=1

We use NLL(f ) := E[− log pf (Y | X)] on the same clean evaluation law. We also use WorstClassAcc(f ) := miny:Pr[Y =y]>0 Pr[ŷf (X) = y | Y = y]. The following paragraphs provide detailed definitions of the OOD, privacy, representation, prediction-disagreement, and corruption coordinates, as well as the distance-fit, normalization, active-coordinate, missing-value, and scalefallback rules used below. In particular, PredDisagree measures top-1 disagreement with the dense reference, and

RobAcc measures corruption accuracy on a corruption distribution. OOD Coordinates. Let pf (y | x) be the predictive distribution, cf (x) := maxy pf (y | x), and ŷf (x) := arg maxy pf (y | x) under a fixed measurable tie-breaking rule. For the OOD coordinates, let sf (x) := −cf (x) be the OOD score, so larger values indicate more OOD-like examples. Clean accuracy is Acc(f ) := Pr[ŷf (X) = Y ]. Then AUROCOOD (f ) is computed for the binary label 1{OOD} with score sf . The orientation-specific indistribution false-alarm coordinate is IDFAR95OOD (f ) := Pr[sf (X) ≥ q0.05 (sf (Z))], where q0.05 is the 5% quantile of the OOD-score distribution. Thus, this coordinate measures in-distribution false alarms at a nominal OOD recall of 95%, induced by this quantile rule. With finite samples and score ties, IDFAR95OOD is the corresponding empirical quantile diagnostic under the OOD-positive scoring convention. The IDFAR95OOD coordinate is distinct from the conventional OOD-detection false-positive rate at a 95% true-positive rate, which treats in-distribution examples as positive and reports OOD acceptance when the in-distribution true-positive rate is 95%. Under the same maximum-confidence score, the latter is Pr[cf (Z) ≥ q0.05 (cf (X))] up to the empirical tie convention. Finally, Conf OOD (f ) := E[cf (Z)]. Policy-Coordinate Direction and Missing Values. two directions of this disagreement are

The

rA→R (f, fD ) := Pr[aD (X) = 1, af (X) = 0], rR→A (f, fD ) := Pr[aD (X) = 0, af (X) = 1]. Suppressing the shared arguments, the two directional rates satisfy AcceptFlipτD = rA→R + rR→A , ∆ReviewRateτD = rA→R − rR→A . Here ∆ReviewRateτD denotes the candidate rate minus the dense-reference rate. Thus, opposite-direction changes can cancel in the aggregate review rate even when many individual inputs are routed differently. The AutoErrτD coordinate is defined when Pr[Af ] > 0. If Pr[Af ] = 0, the coordinate is undefined and reported as missing. Thus, AutoErrτD is the conditional error rate among automatically accepted in-distribution examples, and AcceptFlipτD is the fraction of in-distribution examples whose acceptance or review decision changes relative to the dense reference. Policy-Coordinate Interpretation. Here, review denotes operational abstention or escalation to a human or a downstream verification system; the experiments audit routing decisions and do not assume a particular reviewer or measure the reviewer’s outcome. We also report prediction disagreement with the dense reference. For a service processing N inputs, an acceptance-flip rate ϕ reroutes approximately ϕN inputs even when opposite flip directions leave the net review rate nearly unchanged. AcceptFlipτD measures a different object from a continuous confidence difference. A large confidence change

that remains on the same side of τD leaves the service action unchanged, whereas an arbitrarily small change across τD changes whether that input is automatically accepted or reviewed. The flip rate therefore directly tests compatibility with the fixed deployed gate and complements, rather than replaces, calibration and confidence diagnostics. The two flip directions also have different operational implications: An accept-to-review change can increase review workload and latency, while a review-to-accept change bypasses an escalation required by the dense policy and can change risk exposure. Because AcceptFlipτD pools these directions, the acceptance-flip rate is a bidirectional policy-equivalence diagnostic rather than a signed utility measure. Together, the four policy coordinates—ReviewRateτD , AutoErrτD , OODAcceptτD , and AcceptFlipτD —test whether sparse replacement preserves the operating point of a selectiveclassification or OOD-filtering policy configured for the dense reference and characterize the consequences of failing to preserve that operating point. Privacy Coordinate. The MIAex (f ) coordinate is a compact black-box MIA score reported on the AUC-excess scale. This choice follows the overfitting-based privacy-risk view developed by Yeom et al. (2018), the black-box attack setting introduced by Shokri et al. (2017), and later attack-score refinements (Carlini et al. 2022): Ascore (f ) := max aAUC (s), s∈Sf

1 aAUC (s) := max{AUCmem (s), 1 − AUCmem (s)} − , 2 Sf := {(x, y) 7→ cf (x), (x, y) 7→ −H(pf (· | x)), (x, y) 7→ −ℓY (f ; x, y)}. The reported privacy coordinate is  max{Ascore (f ), Alr (f )}, Alr (f ) ∈ R, MIAex (f ) := Ascore (f ), otherwise, where H is predictive entropy, ℓY (f ; X, Y ) is target-label loss, and Alr is the held-out logistic-regression blackbox attack measured on the same AUC-excess scale. Here AUCmem is computed using the binary membership label 1{member}, with training examples treated as members and test examples as nonmembers. The MIAex coordinate is an AUC excess in [0, 1/2], not a doubled advantage. Representation and Corruption. We use linear centered kernel alignment (CKA) on penultimate representations (Kornblith et al. 2019) and set RepDist(f, fD ) := 1 − CKA(f, fD ) as the representation-dissimilarity coordinate. We also use PredDisagree(f, fD ) := Pr[ŷf (X) ̸= ŷfD (X)]. RobAcc is corruption accuracy on a corruption distribution. Behavioral-Compatibility Distance Details. Let G be the relevant protocol group and let FG be the set of distance-fit rows. These rows are designated by the analysis protocol for estimating coordinate normalization statistics within that group and do not form an additional model class. The designated rows are fixed before scoring candidates and are never

borrowed from a different protocol group. In the reported artifact, FG contains the available dense, small dense, oneshot MP, Staged MP, global random sparse, layerwise random sparse, LT reinitialization, IMP, SNIP, Gradient Signal Preservation (GraSP), Iterative Synaptic Flow Pruning (SynFlow), and RigL rows. LT-baseline and validation-selection rows are not used to fit these normalization statistics. A protocol group is the set of rows sharing the dense-reference identity and seed, dataset, architecture, sparsity, training recipe, candidate-pool, and behavioral-evaluation keys that affect the comparison. For coordinate j, let xj (f ) be the raw candidate value, dj ∈ {−1, 1} be the utility direction of coordinate j, and let µj,G and σj,G be the population mean and standard deviation of the finite values over FG . In the implemented distance, σj,G := 1 when the computed scale is nonfinite or smaller than 10−8 , including the case of a single finite fit value. Define the active coordinate set MG := {j :xj (fD ) is finite and xj (g) is finite for some g ∈ FG }. For j ∈ MG , set  x ej (f ) :=

xj (f ), xj (f ) is finite, µj,G , otherwise,

and Zj (f ) := dj

x ej (f ) − µj,G . σj,G

Together, these active standardized coordinates form the vec−acc tors compared by DBE in the final replacement audit and b −acc,sel in BP-LT selection. by the validation analogue D BE The default equal-coordinate root-mean-square is designed for comparative screening within a protocol. Because the normalization statistics and the active coordinate set are protocol specific, the summary is interpreted primarily within a protocol, and cross-protocol magnitude comparisons are qualitative. The operationally interpretable evidence in our results comes from raw paired policy changes and signed service-level coordinates; the distance summarizes multi-coordinate deviation for within-protocol screening and selection.

Theoretical Results Proof of Theorem 1 Proof. Because PX is nonatomic, for any α ∈ [0, 1] there is a measurable set Tα with PX (Tα ) = α. Define the desired candidate acceptance set by the symmetric difference Sα := AD △ Tα . Choose constants c− ∈ (1/K, τD ) and c+ ∈ (τD , 1), and set  c , x ∈ Sα , cα (x) := + c− , x ∈ / Sα .

For kD (x) := ŷfD (x), define a candidate predictive distribution by  cα (x), y = kD (x), pfα (y | x) := 1 − cα (x)  , y ̸= kD (x). K −1 These measurable, nonnegative probabilities sum to one. Since cα (x) > 1/K, the designated class is the unique maximizer, so ŷfα (x) = ŷfD (x) for every x. The pointwise prediction identity implies equal top-1 accuracy and zero prediction disagreement. The maximum confidence of fα is cα (x), so the candidate’s acceptance set is exactly Sα . Therefore AcceptFlipτD (fα , fD ) = PX (Sα △ AD ) = PX (Tα ) = α. □

The construction proves the theorem. Proof of Proposition 1

Proof. The dense policy accepts exactly when Dτ ≥ 0, whereas the candidate accepts exactly when Dτ + ∆c ≥ 0. The two disjoint ways in which these indicators differ give the equality in Eq. 1. On either event, |∆c | ≥ |Dτ |; equality can occur in the second event when the candidate lies exactly at the threshold. Hence, for every ε > 0, a flip implies either |Dτ | ≤ ε or |∆c | > ε. Taking probabilities proves the stated bound. Markov’s inequality gives E|∆c |p . εp The preceding bounds prove the proposition. Pr[|∆c | > ε] ≤

□

Proof of Theorem 2 Proof. Clipping to [1/K, 1] preserves the side of the interior threshold τD on which the perturbed score lies. Write d = Dτ and partition the joint-density integral by the sign of v. Because the joint law has a density, all boundary events corresponding to the interval endpoints below have probability zero. When v > 0, a flip occurs for −tv ≤ d < 0; when v < 0, a flip occurs for 0 ≤ d < −tv. The joint-density assumption therefore gives pt := AcceptFlipτD (ft , fD ) Z Z 0 Z = h(d, v)dddv + v>0

−tv

v<0

Z −tv h(d, v)dddv. 0

After substituting d = tu and dividing by t, Z Z 0 pt = h(tu, v)dudv t v>0 −v Z Z −v + h(tu, v)dudv. v<0

0

Choose M < ∞ with |V | ≤ M almost surely. The density vanishes almost everywhere outside |v| ≤ M , so the integrals may be restricted to that set. For all sufficiently small t satisfying tM ≤ d0 , continuity at d = 0 makes each inner integral converge to the corresponding interval length times

h(0, v) for almost every v. Moreover, each inner integral is bounded by |v|h̄(v), which is integrable by assumption. Dominated convergence in v therefore yields Z Z pt lim+ = vh(0, v)dv + (−v)h(0, v)dv t t→0 v>0 v<0 Z = |v|h(0, v)dv. R

The dominated convergence argument completes the proof. □ Interpretation of the Theoretical Results The first theorem establishes a structural limitation of accuracy-only certification; the proposition and local theorem then turn that limitation into a candidate-level, testable account of when threshold flips occur. Clean accuracy records only whether the class with the largest predicted probability is correct, whereas the deployed gate tests whether that largest probability is above τD . Theorem 1 therefore says that, for every desired flip rate, one can construct a candidate that keeps every top-1 prediction unchanged yet reverses the accept– review status of that fraction of inputs. The ability to induce an arbitrary flip rate establishes an information limit of the accuracy certificate; empirical candidates are characterized by the following finite-perturbation analysis. Proposition 1 then localizes the risk for a particular candidate: For any chosen tolerance ε, every flip must come either from an input whose dense confidence was already within ε of the threshold or from a confidence change larger than ε. Thus, even uniformly small confidence changes can matter when many inputs are crowded near the deployed threshold. Theorem 2 makes this boundary effect quantitative along a regular local path. For a perturbation of scale t, the flip rate is tκτD + o(t); the threshold-amplification coefficient is large when the dense-confidence distribution has high density at the threshold and the perturbation produces large confidence changes for examples near that boundary. The coefficient is a local sensitivity coefficient, while the finite flip rates from 7% to 10% in our experiments are direct empirical measurements. Operationally, an accuracy-recovery check captures neither this boundary sensitivity nor the tail of larger confidence changes. A policy that reuses a fixed threshold inherited from the dense reference must therefore be re-audited after sparse replacement, motivating the reported AcceptFlipτD coordinate and behavior-aware selection within the accuracy-matched candidate set.

Behavior-Aware Sparse-Candidate Screening To explore whether the conventional LT recovery target can be extended from clean-accuracy matching to closer agreement with the dense incumbent’s measured behavior, we design and evaluate the BP-LT procedure as a validation-time search over a fixed pool of trained sparse candidates. This experiment tests whether behavior-aware selection can identify a more compatible replacement after accuracy recovery. The main text specifies the validation-only selection setup, primary BP-LT in-band selection rule, and parameter values. The subsections below state the scope of this selection

unit and specify the pool contents, candidate eligibility, and restricted-pool variants. Selection Setup Validation-Only Selection Procedure. For each matched selection cell, we first compute the dense and candidate clean b −acc,sel accuracies on held-out validation data and compute D BE from the active behavioral coordinates and held-out diagnostic sources available to that protocol. The normalized validation accuracy gap and validation band are defined in the main text. We then determine which candidates fall inside the band and apply the selection rules below; each rule selects one candidate. Finally, we recompute the reported metrics on the test split. No test metric is used to choose a candidate. The validation band used for selection and the equally wide test recovery band used for final evidence are distinct: Passing the former does not guarantee passing the latter, so test recovery status is reported separately. Selection Rules Accuracy-Only Selection. The accuracy-only baseline chooses the eligible candidate whose finite validation accuracy is closest to that of the dense reference. The validation band determines whether this selection is in-band; if no candidate is in-band, the closest-accuracy selection is flagged as a fallback and the test accuracy gap of the selected candidate is reported. Parameter Settings and Units. Because selection accuracy is represented as a fraction, 0.015 corresponds to 1.5 pp. Throughout, we use % for percentage-valued levels or rates and pp for differences between percentage-valued quantities. These common values define the reported operating rule rather than universal deployment constants; applications can set the band and trade-off to match application-specific replacement contracts. Scope and Selection Unit BP-LT serves as shorthand for behavior-aware validation-time screening of already trained sparse candidates. The default eligible pool can include nonLT sparse methods, and selection is performed with respect to the aggregate compatibility objective; ticket construction, retraining, and coordinate-wise release checks remain separate stages. Selection is performed independently within each matched dense-reference seed, sparsity, and protocol cell. Thus, candidates associated with different dense references, seeds, or training and behavioral-evaluation protocols are never pooled into the same selection problem. Candidate Pool and Eligibility For sparsity s, let Fs := {fi }ni=1 denote the fixed eligible pool for one such matched selection cell. BP-LT selects one existing model from this pool. By default, BP-LT uses the protocol-eligible all-sparse pool. Here, all-sparse means all alternatives designated as selection eligible by that protocol, not every sparse baseline reported in the paper. The standard pools contain LT, LT reinitialization, global random, and layerwise random candidates; larger protocols add eligible IMP, SNIP, or RigL candidates as specified in the experimental protocol. One-shot MP and Staged MP are reported as separate post-training controls

but are not selection eligible, and neither the dense reference nor the small dense control belongs to Fs . The pool and eligibility flags, including the restricted-pool variants below, are fixed before candidate scoring, and the selected source is retained in the accompanying metadata. We also evaluate two restricted variants. Ticket-family BPLT restricts the available candidates to LT, LT reinitialization, IMP, and SNIP when those sources are present, whereas LT-only BP-LT selects only from measured LT-baseline candidates. These variants distinguish gains from broad sparse screening from gains available within progressively narrower ticket pools. Fallback and Edge-Case Rules If the validation accuracy band is empty, BP-LT instead selects fb b −acc,sel (fi , fD ))2 ], fBP ∈ argmin[∆sel i + λ(DBE fi ∈Fs

with deterministic tie-breaking. In the in-band case, ties prefer smaller behavioral distance and then smaller accuracy gap. In the out-of-band case, ties prefer smaller accuracy gap and then smaller behavioral distance. Out-of-band selections are flagged and remain tied to the same replacement criterion, with test accuracy gaps reported explicitly. For BP-LT, sel d clean or D b −acc,sel are inelcandidates with a nonfinite Acc BE igible for these argmin operations. If the dense validation accuracy is nonfinite or no eligible candidate remains, the selection is undefined. BP-LT Selection Implementation Excerpt Listing 1 gives the core validation-time selection rule used for BPLT. The implementation also handles row metadata and restricted candidate pools. The scoring logic selects inside the validation accuracy band when possible and, when the band is empty, minimizes the normalized accuracy gap plus the weighted squared behavioral distance shown above. In the excerpt, test accuracy is never substituted for missing or nonfinite validation accuracy, keeping candidate selection independent of the test split. Listing 1: Python implementation example of BPLT validation-time selection. Each row stores validation accuracy sel_acc, validation behavioralcompatibility distance sel_d_be_no_acc, and a candidate_for_selection flag. 1 import math 2 3 4 def bp_lt_select(rows, dense, acc_band, lam=0.2): 5 def acc(row): 6 return float(row.get("sel_acc", math.nan)) 7 8 def distance(row): 9 value = float(row.get("sel_d_be_no_acc", math 10 11 12 13 14 15 16 17 18

.inf)) return value if math.isfinite(value) else math.inf dense_acc = acc(dense) if not math.isfinite(dense_acc): return None candidates = [ r for r in rows if bool(r.get("candidate_for_selection", False))

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56

and math.isfinite(acc(r)) and math.isfinite(distance(r)) ] if not candidates: return None def gap(row): value = abs(acc(row) - dense_acc) return value if math.isfinite(value) else math.inf in_band = [r for r in candidates if gap(r) <= acc_band] pool = in_band or candidates band = max(float(acc_band), 1e-12) def score(row): g = gap(row) d = distance(row) if in_band: return (d + lam * (g / band) ** 2, d, g) return (g / band + lam * d ** 2, g, d) best = min(pool, key=score) selected = dict(best) selected["method"] = "bp_lt" selected["selected_from"] = best.get("method") rule_in_band = ( "validation_acc_band_plus_" "dense_behavior_distance" ) rule_fallback = ( "fallback_acc_behavior_" "no_in_band" ) selected["selection_rule"] = ( rule_in_band if in_band else rule_fallback ) selected["candidate_for_selection"] = False return selected

Experimental Protocol and Reproducibility Protocol Definitions, Inclusion, and Recovery Criteria An experiment group is the aggregation unit. A protocol group is the behavioral-distance computation unit, defined by a unique combination of dense reference, dense-reference seed, dataset, architecture, sparsity, training recipe, candidate pool, and behavioral-evaluation choices. A selection group is the corresponding validation-time selection problem for a fixed dense reference and selection-eligible candidate pool, so selection-group counts need not equal experiment-group counts. A protocol–method group is a protocol group further split by the reported method. We treat candidate-pool breadth, corruption source, pruning-baseline budget, learning-rate overrides, auxiliary split policy, selection-eligibility pool, and training and evaluation limits as protocol keys. Using these protocol keys keeps specialized protocols with stronger baselines in the corresponding protocol groups. Hardware, group definitions and required-run inclusion criteria, exact dataset–architecture– sparsity cells, candidate eligibility and counts, training recipes, auxiliary-data coverage, and statistical summaries are fixed by the reported protocol before model aggregation. We verify compliance with prespecified experiment-group inclusion criteria before aggregation. Within each experiment group, every required seed–sparsity cell must contain the dense reference, one-shot MP, an LT-baseline candidate, and variants produced by the validation selection rules, together with any protocol-specific strong baselines such as SNIP, IMP, RigL, or Staged MP. These criteria concern the

availability of required model rows rather than metric values or recovery outcomes; all 56 experiment groups satisfy the inclusion criteria. We stratify each protocol–method group by recovery quality. A group provides study-band-matched evidence under our strict study-defined criterion when the group’s dense reference passes the dataset-specific accuracy floor and every evaluated accuracy gap in the group is finite and lies within the ±1.5 pp test recovery band. Here, the strict label means that every evaluated gap in the protocol–method group satisfies the rule; raw gaps remain reported so that narrower application-specific margins can be applied. This test recovery band differs in both role and data split from the validation accuracy band used by the selection rules. Both bands have a 1.5 pp tolerance, but we use the former to classify test evidence and the latter to screen candidates on validation data. This ±1.5 pp band is a prespecified stratification convention, not a universal equivalence margin or an application release tolerance. The dense-reference accuracy floors are 93% for CIFAR-10, 72% for CIFAR-100, 85% for Imagenette, 90% for Flowers-102, and 75% for FGVC-Aircraft. We place a group in the near-recovery stratum if the group’s dense reference passes the accuracy floor, the group fails this strict criterion, and the group’s absolute mean accuracy gap is at most 3 pp. Other evaluated sparse protocol–method groups that fail the dense-reference accuracy floor or the remaining recovery criteria form recovery-stress regimes. These regimes characterize how behavioral gaps evolve as sparse recovery degrades. This stratification separates the accuracymatched replacement audit from the transition regimes where sparse-recovery quality changes. The term accuracy-matched below denotes a regime or setting that satisfies this study-defined rule; the term is not an application-independent equivalence claim. Near-recovery and recovery-stress regimes are identified explicitly. We base policy-level statements on evaluations with the corresponding policy coordinates. We compute aggregate behavioral distances from coordinates with finite dense-reference values and at least one finite distance-fit value in the protocol group. Active coordinates missing for a candidate are imputed with the protocol distance-fit mean. Diagnostics unmeasured at the protocol level remain inactive for that protocol. Datasets, Architectures, and Evaluation Sources We conduct the primary evaluation on CIFAR-10 with a CIFARstyle ResNet-18 (Krizhevsky, Hinton et al. 2009; He et al. 2016) at 50%, 80%, 90%, and 95% sparsity. We also evaluate CIFAR-100 with ResNet-18 at 50% and 80% sparsity, CIFAR-10 with ResNet-34 at 80% sparsity, and CIFAR-10 with WideResNet-28-2 (Zagoruyko and Komodakis 2016) at 80% sparsity for architecture replication; replication counts for these settings are reported in the Seed and Candidate-Pool Details subsection below. The standard CIFAR protocols use SVHN (Netzer et al. 2011) as the OOD dataset. The same experimental setup also supports the 10-class self-taught learning dataset (STL10) (Coates, Ng, and Lee 2011), CIFAR-100, and directorystructured OOD datasets such as Tiny-ImageNet. The largepool CIFAR-10 protocol merges a broader OOD pool from

SVHN, CIFAR-100, and STL-10 with all 15 corruption families in CIFAR-10-C at severities 1, 3, and 5 under a 10,000example evaluation cap. CIFAR-10-C corruptions are used for CIFAR-10 corruption-accuracy evaluation when available. Imagenette is an ImageNet-derived, higher-resolution image-classification benchmark (Deng et al. 2009). We use Imagenette for ConvNeXt-Tiny (Liu et al. 2022) and ViT-Tiny (Dosovitskiy et al. 2021) at 50% sparsity with pretrained PyTorch Image Models (timm) backbones (Wightman 2019) and no small dense baseline. The OOD and corruption selection splits are disjoint from the corresponding evaluation splits. These pretrained and transfer settings follow evidence that LT structure can persist after vision pretraining and that sparse ImageNet models can transfer to downstream tasks (Chen et al. 2021; Iofinova et al. 2022). The fine-grained transfer suite contains three evaluations of pretrained ConvNeXt-Tiny models on 224-pixel inputs. Flowers-102 (Nilsback and Zisserman 2008) at 80% sparsity is a recovery-stress setting. Flowers-102 at 50% sparsity is a transfer protocol without a small dense baseline. FGVC-Aircraft (Maji et al. 2013) at 50% sparsity is a transfer protocol. Auxiliary split status is part of the protocol key for transfer experiments. The Flowers-102 50% protocol is marked as disjoint, whereas the FGVC-Aircraft protocol has no recorded disjoint-split designation and is analyzed separately. Base Training Recipe Comparisons use protocol-matched training budgets in both from-scratch and pretrained regimes. The dense references and candidates from the LT baseline, LT reinitialization, global random sparse, and layerwise random sparse methods are trained from scratch for 200 epochs with stochastic gradient descent, a base learning rate of 1 × 10−1 , momentum 0.9, Nesterov updates, weight decay 5 × 10−4 , random cropping and flipping, a batch size of 512, bfloat16 autocast, a 5-epoch warmup, and cosine learning-rate decay. One-shot MP candidates start from the trained dense reference, apply the one-shot global MP mask, and fine-tune for 30 epochs on CIFAR-10 and 40 epochs on CIFAR-100. We report active parameter counts for sparse networks. CIFAR-10 ResNet-18 has 11.17 million dense parameters and 5.59, 2.24, 1.13, and 0.57 million active parameters at 50%, 80%, 90%, and 95% sparsity. CIFAR-10 WideResNet28-2 has 1.47 million dense parameters and 0.30 million active parameters at 80% sparsity. Following a common convention in unstructured weight pruning (Gale, Elsen, and Hooker 2019; Tanaka et al. 2020), pruning masks are applied only to trainable convolutional and linear weight tensors. Biases, normalization parameters, and non-weight state are left dense; this convention is important for transformer experiments, where positional embeddings and class tokens are kept outside the ordinary prunable weight matrices. Global MP, IMP, SNIP, GraSP, and SynFlow masks are constructed by selecting the exact global top-k flattened indices from the pruning scores, rather than by thresholding at the cutoff. This exact top-k selection prevents score ties, which can occur during pruning at initialization, from silently

changing the requested number of active maskable weights. When included, the RigL-style baseline is trained with the same sparse training budget as other sparse candidates trained from initialization. Protocol groups distinguish both the presence of this baseline and the number of topology updates actually performed. Modern and Transfer Recipes For the Imagenette protocols with 224-pixel inputs, the ConvNeXt-Tiny and ViT-Tiny backbones are initialized with pretrained weights provided by timm, while both classifier heads are initialized randomly. Both models are then trained for 100 epochs using AdamW with a learning rate of 2 × 10−4 , weight decay 5 × 10−2 , label smoothing 0.1, exclusion of normalization parameters and biases from weight decay, gradient clipping at 1.0, and cosine learning-rate decay. The LT rewinding point consists of the pretrained backbone and the classifier head’s initial random parameter values. The Imagenette OOD pool is a disjoint union of CIFAR-100 and Tiny-ImageNet splits, resized to 224 pixels and normalized with ImageNet statistics; corruption accuracy is evaluated under deterministic Gaussian noise applied to held-out Imagenette test and validation images. The Flowers-102 and FGVC-Aircraft transfer protocols follow the same pretrained ConvNeXt-Tiny recipe as the Imagenette protocol, with 100 epochs for dense references and sparse candidates and a 20-epoch MP fine-tuning run. Candidate Pools and Sparse Baselines Baseline definitions and selection eligibility. Throughout the experiments, an LT baseline is a sparse candidate obtained by applying the associated LT mask and training the resulting subnetwork from the protocol-specific rewinding point. The rewinding point is the original random initialization in from-scratch protocols or the pretrained backbone with the classifier head restored to the initially sampled random parameter values in pretrained protocols. Tables label these rows LT. LT reinitialization trains the same LT mask from a separately sampled random initialization; tables abbreviate this method as LT reinit. One-shot MP denotes post-training global MP followed by fine-tuning; tables label this baseline One-shot MP. Staged MP denotes the separate multi-stage post-training MP baseline. The labels Random sparse and Layerwise random denote, respectively, global random sparse masks and random masks that preserve the corresponding LT mask’s per-layer sparsity. Small dense denotes a fully dense, fixed-width convolutional network with base width 48, trained using the protocol’s dataset-specific recipe. The Small dense model is a lower-capacity baseline rather than a sparsified version of the dense reference; tables label this row Small dense. Candidate eligibility is distinct from baseline reporting. The default all-sparse pool comprises the sparse alternatives marked selection eligible for each protocol, and larger protocols broaden both the sparse-baseline set and the eligible pool. Post-training baselines include one-shot MP and Staged MP when the latter is present. These post-training baselines, small dense baselines, and dense references are reported sep-

arately and are not selection eligible. Protocol-level candidate accounting is given in the Seed and Candidate-Pool Details subsection. All candidates are selected using held-out validation metrics; we recompute all metrics on the test split for the final tables. Selection groups without an in-band candidate are flagged explicitly, and final tables report the resulting test accuracy gap together with the behavioral coordinates. Large-pool sparse baselines. The large-pool CIFAR-10 ResNet-18 protocol at 50% sparsity expands the selectioneligible pool with multiple candidates from the LT baseline, LT reinitialization, global random sparse, layerwise random sparse, IMP, and SNIP methods (Han et al. 2015; Lee, Ajanthan, and Torr 2019). The candidate composition is listed in the Seed and Candidate-Pool Details subsection below. The evaluation framework also supports GraSP and datafree SynFlow candidate types for pruning at initialization (Wang, Zhang, and Grosse 2020; Tanaka et al. 2020), as well as RigL-style dynamic sparse training candidates (Evci et al. 2020). GraSP and SynFlow are not part of the aggregate comparisons reported here. We measure the 200-epoch dynamic sparse training baseline in the CIFAR-10 ResNet-18 RigL protocol at 80% sparsity, using the same sparse budget as the LT candidates. The aggregate 200-epoch comparisons reported here focus on the measured RigL baseline. The RigL procedure starts from a global random sparse mask, periodically drops low-magnitude active weights, regrows inactive weights with large instantaneous gradients, and preserves the exact global active weight count. Protocol variants that change pruning fine-tuning learning rates or add the Staged MP baseline are tracked as separate protocol groups. Selection-eligibility pool variants. The restricted-pool analysis reruns validation selection with ticket-family and LT-only eligibility criteria. The analysis quantifies behavioraware selection for the operationally defined ticket-family or LT-only candidate sets. The ticket-family pool contains candidates from the LT baseline, LT reinitialization, IMP, and SNIP methods. The LT-only variant limits reselection to LT-baseline candidates. We report these reselection results alongside the primary all-sparse analysis. These post-hoc reselection outputs are derived rows and are not added to the primary artifact’s 846-row accounting. The result-quality summary reports, for every protocol– method group, the fraction of evaluations within the ±1.5 pp test recovery band, the maximum absolute accuracy gap, and an indicator that all evaluations in that group satisfy the band. A non-strict group with an absolute mean accuracy gap of at most 3 pp is labeled near-recovery. These summaries distinguish strict accuracy-matched replacement evidence, near-recovery, and recovery-stress regimes. Separate protocol keys specify the learning rates for densereference training, sparse-candidate training, post-pruning fine-tuning, and IMP. This protocol keying keeps stronger pruning-baseline sweeps in protocol groups keyed by the group-specific training and fine-tuning recipes.

Seed and Candidate-Pool Details Unless otherwise stated, candidate counts below are per matched dense seed– sparsity cell, whereas table rows aggregate measured models over the dense seeds specified for each protocol. Accuracyonly and BP-LT rows are derived by selecting an already measured candidate for each dense reference; the derived rows add one reported row per selection group but no additional training run. Model fields and reseeding. Each measured model entry records the dataset, architecture, sparsity, dense seed, candidate training seed, mask seed, and candidate source; a derived selection row additionally records the selected source. We use these fields together with the protocol keys defined above to assign training runs, behavioral-distance fits, and selection problems to the appropriate groups. The dense seed identifies the matched reference, the candidate training seed controls stochastic candidate training, and the mask seed records the seed used for mask construction when applicable. For entries without a measured diagnostic, the corresponding field is left blank; optional identifiers such as mask seeds are retained when applicable. Evaluation transforms for validation selection are deterministic. For protocols that define disjoint auxiliary selection and evaluation splits, those split assignments are fixed explicitly. Default sparse-candidate pools. For CIFAR-10 ResNet18 and CIFAR-100 ResNet-18, the default selection-eligible pool contains six sparse candidates per dense seed–sparsity cell: one LT-baseline candidate trained from the original random initialization with the LT mask, one candidate trained with the same mask after random reinitialization, two candidates with global random sparse masks, and two candidates with layerwise random sparse masks that have the same perlayer sparsity as the corresponding LT mask. For CIFAR10 ResNet-34 and CIFAR-10 WideResNet-28-2 (Zagoruyko and Komodakis 2016), the pool contains four candidates, one from each of those four method types. Separately reported one-shot MP and small dense rows are not included in these pool counts. When the restricted-pool analysis is limited to LT-baseline candidates, improvements depend on variation across the measured LT-candidate training seeds. Large-pool and RigL protocols. At 50% sparsity, the large-pool CIFAR-10 ResNet-18 evaluation comprises three dense seeds and 15 selection-eligible candidates per dense seed: three LT-baseline training seeds, two LT reinitializations, four candidates with global random sparse masks, four candidates with layerwise random sparse masks, one IMP candidate, and one SNIP candidate. Accordingly, Appendix Table 6 aggregates nine, six, twelve, twelve, three, and three measured candidates for these methods, respectively, along with three dense-reference runs. The three separately reported One-shot MP rows are not part of the 15-candidate eligible pool, and each selection rule contributes three derived rows. At 80% sparsity, the large-pool evaluation also includes three dense seeds. For each dense seed, the selection-eligible pool contains one LT-baseline candidate, one LT reinitialization, one candidate with a global random sparse mask,

one candidate with a layerwise random sparse mask, one SNIP candidate, and one two-round IMP candidate with 20 fine-tuning epochs per round. Table 2 therefore aggregates three measured models per method and three derived rows per selection rule. One-shot MP and the four-stage Staged MP baseline are reported separately and are not selection eligible. For the CIFAR-10 ResNet-18 RigL protocol at 80% sparsity, we retain the same three dense seeds. RigL performs 132 topology updates per seed at the specified update interval and preserves the exact global active weight count. In the companion protocol with a lower post-pruning fine-tuning learning rate, LT and RigL are selection eligible, whereas one-shot MP and Staged MP are reported separately. Appendix Table 7 aggregates three measured models for each method and three derived rows for each selection rule. Replication counts. The primary CIFAR-10 ResNet-18 evaluation covers four sparsity levels (50%, 80%, 90%, and 95%) for five dense seeds. CIFAR-100 ResNet-18 at the 50% and 80% sparsity levels, CIFAR-10 ResNet-34 at 80% sparsity, and CIFAR-10 WideResNet-28-2 at 80% sparsity are evaluated for three dense seeds. The ConvNeXt-Tiny (Liu et al. 2022) and ViT-Tiny (Dosovitskiy et al. 2021) Imagenette protocols at 50% sparsity are evaluated with three dense seeds and, for each dense seed, two training seeds for LT-baseline candidates; the LT entries therefore each summarize six candidates. The Flowers-102 (Nilsback and Zisserman 2008) recovery-stress protocol at 80% sparsity, the Flowers-102 protocol at 50% sparsity, and the FGVCAircraft (Maji et al. 2013) protocol at 50% sparsity are likewise evaluated with three dense seeds and two LT training seeds per dense seed. The LT entries for these three protocols therefore each summarize six candidates, whereas the dense, accuracy-only, BP-LT, and one-shot MP entries each summarize three. In both 50% transfer protocols, the LT, accuracy-only, BP-LT, and one-shot MP protocol–method groups satisfy the study-defined accuracy-matching criterion; the random-mask controls do not. Behavioral Evaluation Coverage Corruption protocol coverage. Each corruption-accuracy evaluation records the evaluated corruption source explicitly. The recorded source distinguishes benchmark corruption data from synthetic Gaussian-noise stress tests; the benchmark data come from the CIFAR corruption family or directory-structured corruption datasets. In the standard CIFAR-10 rows with CIFAR-10-C coverage, we use Gaussian noise, shot noise, brightness, and contrast corruptions at severities 1, 3, and 5. For the large-pool CIFAR-10 protocols, we evaluate all 15 CIFAR-10-C corruption families at the same severities. For CIFAR-100, RobAcc denotes corruption accuracy under the Gaussian-noise stress test used in those rows. Policy-coordinate availability. The ReviewRateτD , AutoErrτD , and OODAcceptτD coordinates are available for the Imagenette ConvNeXt and ViT protocols, the fine-grained transfer protocols, and the large-pool CIFAR-10 protocols. The AcceptFlipτD and PredDisagree

Statistical Summaries We report means and standard deviations across replicated evaluations, emphasize paired effect sizes and replication across sparsities and architectures, and include sensitivity analyses that apply rank distance and shrinkage-based Mahalanobis distance to the same behavioral-coordinate comparisons used by the diagonal distance. We treat these summaries as replicated descriptive evidence, not as formal statistical equivalence or non-inferiority tests. An operational release analysis should attach uncertainty intervals to predeclared coordinate-specific tolerances. Protocol Accounting and Compute Experiments were run on one Aurora compute node with six graphics processing units (GPUs), specifically Intel Data Center GPU Max 1550 devices, each comprising two independently addressable GPU tiles. The node therefore provides 12 tiles in total. We used PyTorch’s torch.xpu backend, and each training process was assigned to one tile via ZE_AFFINITY_MASK. The aggregate artifact contains 846 result rows from 56 experiment groups, comprising 624 non-selection rows and 222 derived validation-selection rows. A selection row carries the metrics of the selected candidate, so 846 is not the number of independent training runs. Method identity and protocol fields that encode choices affecting training or behavioral evaluation index each row.

Supplemental CIFAR-10 Evidence CIFAR-10 ResNet-18 Trade-Off Plot Appendix Figure 4 shows the same results in a trade-off plot. The 50% and 80% sparsity levels are the cleanest replacement regimes. Several sparse methods are close to zero accuracy gap, yet the corresponding behavioral distances remain visibly above zero. The 90% and 95% sparsity levels show the high-sparsity regime where accuracy recovery starts to degrade. Per-Metric Behavioral Decomposition Aggregate distances are useful for selection, but the distance summaries can hide which behavioral coordinates move. Appendix Table 4 reports signed test-set differences from the dense reference for representative settings. The sign convention follows each coordinate’s raw orientation. For example, lower NLL and lower IDFAR95OOD are improvements, whereas a positive representation distance indicates that the sparse candidate’s representation differs from that of the dense reference. The key point is that accuracy-matched LT baselines move several deployment-relevant behavioral coordinates at once. For CIFAR-10 ResNet-18 at 50% sparsity, the LT baseline improves OOD AUROC and corruption accuracy slightly

50% Sparsity

80% Sparsity

2.0 1.5 1.0 0.5 −acc DBE

coordinates are available for the ConvNeXt protocol at 50% sparsity, the ViT protocol at 50% sparsity, the Flowers-102 transfer protocol at 50% sparsity, the FGVC-Aircraft transfer protocol, and the large-pool CIFAR-10 protocols. A coordinate enters the standardized distance when the dense reference value is finite. The coordinate must also be finite for at least one distance-fit row in the same protocol group. Coordinates not measured at the protocol level are inactive for that protocol. If a row is missing an otherwise active coordinate, we impute that coordinate with the protocol distance-fit mean before standardization.

−0.3

−0.2

−0.1

0.0

−1.00

−0.75

90% Sparsity

−0.50

−0.25

0.00

95% Sparsity

2.0 1.5 1.0 0.5 −2.0

−1.5

−1.0

−0.5

0.0

−4

−3

−2

−1

0

Accuracy gap from dense reference (points) LT Acc-only

BP-LT MP

Reinit Random

Layerwise

Figure 4: CIFAR-10 ResNet-18 accuracy gap and cleanaccuracy-excluded behavioral-compatibility distance. Markers show method means over evaluated models with small display offsets to separate overlapping methods; the vertical reference line marks the dense reference’s zero accuracy gap, and the shaded region marks the ±1.5 pp test recovery band.

but changes representation geometry and IDFAR95OOD . At 80% sparsity, the LT baseline has similar accuracy but lower corruption accuracy. The WideResNet results show larger raw shifts because sparse recovery is harder. Controls at 80% and 95% Sparsity Levels Appendix Table 5 gives additional CIFAR-10 ResNet-18 controls. At 80%, one-shot MP fine-tuning reaches 94.26% accuracy and a distance of 0.821, LT reinitialization reaches a distance of 0.825, and candidates with layerwise random sparse masks have a mean distance of 0.818. These controls are all accuracycompetitive but behaviorally distinct from the dense reference on the measured audit coordinates. At 95%, one-shot MP fine-tuning gives the best clean accuracy among sparse controls, but the behavioral distance of one-shot MP is larger than the distances of the LT baseline and BP-LT. This 95% control result illustrates why we separate accuracy matching from behavioral matching. Expanded 50% Pool with IMP and SNIP The primary CIFAR-10 evaluation in Table 1 is based on 200-epoch training with a compact sparse selection-eligible pool. We next test CIFAR-10 ResNet-18 at 50% sparsity with a broader sparse set and auxiliary evaluation suite. Candidate composition and auxiliary-source details are in the Experimental Protocol and Reproducibility subsection above. Appendix Table 6 shows that the larger selection-eligible pool makes clean-accuracy recovery easier, not harder. The dense reference reaches 94.55% accuracy. The LT-baseline candidates average 94.70%. LT reinitialization, IMP, SNIP, global random sparse, layerwise random sparse, and one-shot MP baselines all remain within about 0.16 pp of the dense reference. Together, these groups provide study-band-matched evidence of measurable behavioral deviation. The LT base-

Setting

Method

∆ECE

∆NLL

(pp)

∆OOD

∆IDFAR95OOD

AUROC

(pp)

∆RepDist

∆RobAcc

∆Worst

(pp)

(pp)

ResNet-18 50%

LT BP-LT

−0.30 −0.17

−0.0126 −0.0061

+0.0102 +0.0013

−9.03 −5.10

+0.0343 +0.0366

+0.16 −0.58

+0.22 +0.14

ResNet-18 80%

LT BP-LT

−0.01 +0.06

+0.0040 +0.0114

+0.0027 −0.0161

−6.59 +3.09

+0.0441 +0.0470

−1.57 −1.94

+0.12 −0.22

WideResNet-28-2 80%

LT BP-LT

+0.71 +0.65

+0.0551 +0.0535

−0.0535 −0.0243

+17.91 +6.50

+0.0950 +0.0915

−3.54 −2.86

−1.30 −2.07

Table 4: Per-metric signed candidate-minus-reference differences. Lower ECE, NLL, and IDFAR95OOD are better, whereas higher OOD AUROC, RobAcc, and WorstClassAcc are better. A positive RepDist means that the candidate is farther from the dense reference in representation space. Worst denotes WorstClassAcc.

Acc (%)

∆Acc (pp)

−acc DBE

80

One-shot MP LT reinit Layerwise random Small dense

94.26 94.35 94.39 88.05

−0.36 −0.27 −0.23 −6.57

0.821 0.825 0.818 2.851

95

One-shot MP LT reinit Layerwise random Small dense

93.76 93.28 93.30 88.05

−0.86 −1.34 −1.32 −6.57

1.313 1.137 1.126 2.801

Sparsity (%)

Method

Table 5: CIFAR-10 ResNet-18 controls. Method

Acc (%)

∆Acc (pp)

−acc DBE

OOD AUROC

Flip (%)

RobAcc (%)

Dense

94.55 ± 0.18

0.00

0.000

0.8688

0.00

71.75

LT LT reinit IMP SNIP Random sparse Layerwise random One-shot MP

94.70 ± 0.16 94.62 ± 0.12 94.50 ± 0.23 94.50 ± 0.07 94.41 ± 0.21 94.62 ± 0.11 94.52 ± 0.06

+0.16 +0.07 −0.05 −0.05 −0.14 +0.08 −0.03

0.952 0.989 1.096 1.074 0.957 1.027 1.054

0.8630 0.8648 0.8599 0.8538 0.8586 0.8574 0.8542

8.29 8.05 8.40 8.12 8.49 8.33 7.22

71.85 71.83 72.11 71.55 71.52 72.13 71.73

Accuracy-only BP-LT

94.64 ± 0.04 94.50 ± 0.05

+0.09 −0.04

0.958 0.832

0.8650 0.8667

8.13 8.29

71.95 71.47

Small dense

88.17 ± 0.19

−6.38

3.925

0.7799

21.90

60.50

Table 6: Large-pool CIFAR-10 ResNet-18 at 50% sparsity. −acc line has DBE = 0.952, prediction disagreement 4.5%, and AcceptFlipτD = 8.3% for decisions made using the dense threshold. BP-LT reduces the behavioral distance to 0.832 while preserving the dense reference’s accuracy. The BPLT selections draw from LT reinitialization, global random sparse, and SNIP candidates, so this row evaluates behavioraware sparse selection. The LT rows isolate the behavioral incompatibility of LTs.

RigL and Staged MP with Lower Fine-Tuning Learning Rate A companion CIFAR-10 ResNet-18 protocol at 80% sparsity strengthens the dynamic sparse training and posttraining pruning baselines with one LT-baseline candidate, one RigL-style dynamic sparse training candidate, a oneshot MP baseline, and a Staged MP baseline. This protocol keeps the sparse-candidate learning rate at 1 × 10−1 and sets the post-pruning fine-tuning learning rate to 1 × 10−2 ; the learning rates for sparse training and post-pruning fine-tuning

∆Acc (pp)

−acc DBE

Method

Acc (%)

Flip (%)

PredDis (%)

RobAcc (%)

Dense

94.66 ± 0.07

0.00

0.000

LT One-shot MP Staged MP RigL

94.50 ± 0.17 94.52 ± 0.20 94.58 ± 0.15 94.50 ± 0.07

−0.16 −0.14 −0.07 −0.16

2.407 1.736 1.577 2.021

0.8684

0.00

0.00

72.24

0.8599 0.8662 0.8663 0.8614

8.54 3.68 3.23 8.93

4.44 1.71 1.58 4.83

71.32 71.05 71.56 71.28

Accuracy-only BP-LT

94.46 ± 0.09 94.50 ± 0.07

−0.19 −0.16

2.045 2.021

0.8623 0.8614

8.80 8.93

4.68 4.83

71.42 71.28

OOD AUROC

Table 7: CIFAR-10 ResNet-18 at 80% sparsity with RigL and Staged MP under the companion protocol with a lower postpruning fine-tuning learning rate. LT and RigL candidates are trained for 200 epochs under the sparse-training recipe; pruning baselines follow the corresponding protocol-specific fine-tuning schedules. PredDis denotes PredDisagree.

define separate protocol keys. The LT and RigL candidates are trained with the same 200-epoch sparse training budget. RigL follows the specified update interval and preserves the exact global active weight count. Appendix Table 7 shows that alternative baseline hyperparameters change the post-training pruning picture, while the replacement conclusion remains unchanged. Staged MP now recovers the dense reference’s accuracy, reaching 94.58% compared with 94.66% for the reference, and is the sparse baseline closest in behavior to the dense reference in this −acc protocol, with DBE = 1.577, PredDisagree = 1.6%, and AcceptFlipτD = 3.2%. RigL also recovers clean accuracy at 94.50% but shows a larger behavioral distance of 2.021, PredDisagree = 4.8%, and AcceptFlipτD = 8.9%. The LT baseline is similarly accuracy-matched but farther from the dense reference’s behavior in this protocol, with −acc DBE = 2.407. Thus, even a candidate produced by strong dynamic sparse training remains behaviorally distinguishable from the dense reference after clean accuracy recovers.

Replications and Transfer This subsection gives the full tables and per-setting interpretation for the replication and transfer results summarized in the main text. Appendix Table 8 reports full CIFAR-100, ResNet-34, and WideResNet replication results, and detailed interpretation of the pretrained settings is given below. The replications reproduce the pattern of accuracy recovery with behavioral gaps across changes in datasets and architec-

tures. Appendix Table 9 provides the pretrained replication. ConvNeXt-Tiny and ViT-Tiny both recover the dense reference’s accuracy at 50% sparsity while retaining nonzero behavioral distance and dense-threshold acceptance flips. CIFAR-100 and Architecture Replications Appendix Table 8 reports the harder CIFAR-100, deeper ResNet-34, and WideResNet settings. CIFAR-100 at 50% sparsity is the most important additional accuracy-matched result. The dense reference reaches 75.30%, the LT baseline reaches 74.66%, and BP-LT reaches 74.95%. The behavior-aware selection remains closer in behavior than the LT baseline. At 80% sparsity, CIFAR-100 still shows the behavioral pattern as recovery becomes harder. The LT baseline and BP-LT are 1.55 and 1.50 pp, respectively, below the dense reference on average. Some individual evaluation gaps exceed the prespecified ±1.5 pp test recovery band, but the mean gaps remain within the near-recovery threshold of 3 pp; these protocol–method groups are therefore classified as near-recovery. CIFAR-10 ResNet-34 at 80% sparsity replicates the main phenomenon with a stronger dense reference. The LT baseline is within 0.07 pp of the reference but has a behavioral distance of 1.029. BP-LT lowers the behavioral distance to 0.574 while remaining within 0.31 pp of the reference. CIFAR-10 WideResNet-28-2 is a harder architecture stress case. The dense reference reaches 94.18%, while the 80% LT baseline reaches 93.01%. This WideResNet LT baseline remains within the prespecified ±1.5 pp test recovery band with a larger 1.17 pp gap. The LT baseline is behaviorally distinct from the corresponding dense reference on the mea−acc sured audit panel, with DBE = 1.149. BP-LT lowers this distance to 1.014. This result shows why architecture diversity, band status, and the actual accuracy gap should be reported together: Recovery can be marginal even when the test recovery band used in this paper is satisfied. Detailed ConvNeXt and ViT Results ConvNeXt Recovery with Disjoint Auxiliary Splits. To evaluate behavioral compatibility with a strong pretrained dense reference, we evaluate an Imagenette ConvNeXt-Tiny protocol at 50% sparsity using 224-pixel inputs and disjoint auxiliary selection and evaluation splits. The pretrained training recipe and candidate composition are described in the Experimental Protocol and Reproducibility subsection above. The ConvNeXt rows in Appendix Table 9 show the result. The dense reference reaches 97.49% accuracy. The LT baseline at 50% sparsity reaches 97.94% mean test accuracy, slightly above the dense reference, yet the LT baseline’s clean-accuracy-excluded behavioral-compatibility distance remains 1.205. BP-LT selects a candidate that is closer under the validation criterion, reducing the distance to 0.974 while maintaining 97.90% accuracy. Accuracy-only selection chooses the candidate with the closest validation accuracy but is behaviorally farther from the dense reference, with a distance of 1.436. One-shot MP fine-tuning almost exactly recovers the dense reference’s accuracy and has a distance of 0.716, showing that a conventional pruning baseline can be

Method

Acc (%)

∆Acc (pp)

−acc DBE

RobAcc (%)

0.000 0.768 0.548

43.50 42.10 42.29

0.000 1.005 0.894 1.243

43.50 40.06 40.51 42.09

0.000 1.029 0.723 0.574 0.483

67.75 68.29 67.50 66.56 68.12

0.000 1.149 1.131 1.014 1.071

65.21 61.67 61.45 62.34 64.12

CIFAR-100 ResNet-18, 50% sparsity Dense LT BP-LT

75.30 74.66 74.95

0.00 −0.64 −0.35

CIFAR-100 ResNet-18, 80% sparsity Dense LT BP-LT One-shot MP

75.30 73.75 73.80 74.55

0.00 −1.55 −1.50 −0.75

CIFAR-10 ResNet-34, 80% sparsity Dense LT Accuracy-only BP-LT One-shot MP

94.86 94.79 94.70 94.54 94.74

0.00 −0.07 −0.15 −0.31 −0.11

CIFAR-10 WideResNet-28-2, 80% sparsity Dense LT Accuracy-only BP-LT One-shot MP

94.18 93.01 92.94 93.04 93.53

0.00 −1.17 −1.24 −1.14 −0.65

Table 8: Results for additional dataset and architecture settings. Each entry is the mean over evaluated models. Method

Acc (%)

∆Acc (pp)

−acc DBE

OOD AUROC

Flip (%)

RobAcc (%)

ConvNeXt-Tiny Dense

97.49 ± 0.14

0.00

0.000

0.843

0.00

96.60

LT Accuracy-only BP-LT One-shot MP

97.94 ± 0.11 97.98 ± 0.15 97.90 ± 0.06 97.47 ± 0.08

+0.45 +0.49 +0.42 −0.02

1.205 1.436 0.974 0.716

0.829 0.785 0.873 0.895

9.69 9.75 9.62 11.74

97.28 97.29 97.27 96.52

Random sparse Layerwise random

87.50 ± 0.04 81.69 ± 0.92

−9.99 −15.80

1.577 2.236

0.758 0.732

34.37 49.26

84.32 81.07

ViT-Tiny Dense

97.01 ± 0.32

0.00

0.000

0.833

0.00

96.51

LT Accuracy-only BP-LT One-shot MP

96.73 ± 0.12 96.75 ± 0.16 96.67 ± 0.17 96.06 ± 0.06

−0.28 −0.26 −0.34 −0.95

0.422 0.432 0.376 0.464

0.861 0.857 0.854 0.776

8.08 8.21 8.08 11.02

95.82 95.88 95.66 95.03

Random sparse Layerwise random

83.54 ± 0.29 81.24 ± 0.37

−13.47 −15.77

1.975 2.130

0.768 0.737

52.08 53.35

81.93 80.30

Table 9: Imagenette evaluations with pretrained ConvNeXtTiny and ViT-Tiny at 50% sparsity on 224-pixel inputs. Auxiliary selection and evaluation splits are disjoint, and corruption accuracy is evaluated under the Imagenette Gaussiannoise protocol.

closer in behavior in this particular setting. The global random sparse and layerwise random sparse controls show that active parameter count alone does not explain the ConvNeXt behavioral gap. Both controls are trained for the same 100 epochs but reach 87.50% and 81.69% test accuracy, respectively, with much larger AcceptFlipτD and PredDisagree rates. The ConvNeXt rows in Appendix Table 9 show that recovery stress does not explain the behavioral gap. Here the

LT baseline exceeds the dense reference’s clean accuracy, but replacing the dense reference still flips about 9.7% of dense-threshold acceptance decisions and disagrees with the dense reference’s predictions on about 2.5% of test examples. The combination of higher clean accuracy with policy and prediction shifts is the operating regime addressed by our behavioral- compatibility audit. ViT Recovery at 50% Sparsity. The pretrained ViT-Tiny evaluation extends the recovery pattern to transformer backbones evaluated at 50% sparsity using 224-pixel inputs and disjoint auxiliary selection and evaluation splits. The ViT rows in Appendix Table 9 report the result. The dense reference reaches (97.01 ± 0.32)%, and the LT baseline reaches (96.73 ± 0.12)%, a 0.28 pp mean accuracy gap. Thus, 50% sparsity constitutes a transformer-backed setting matched under the study-defined accuracy rule. The recovered LT baseline for ViT-Tiny in Appendix Table 9 is still not behaviorally interchangeable with the dense −acc reference: DBE = 0.422, AcceptFlipτD is about 8.1%, and top-1 predictions disagree on about 2.9%. BP-LT selects a candidate closer in behavior, reducing the distance to 0.376 while keeping accuracy at 96.67%. One-shot MP fine-tuning is also strong in clean accuracy at 96.06% but has a larger behavioral distance and larger AcceptFlipτD rate. Global random sparse and layerwise random sparse controls achieve test accuracies of 83.54% and 81.24%, respectively. The ViT result therefore adds a transformer-backed case matched under the study-defined accuracy rule. Once recovery occurs, clean accuracy still does not imply behavioral compatibility. Fine-Grained Transfer Appendix Table 10 reports finegrained transfer evaluations of pretrained ConvNeXt-Tiny models on 224-pixel inputs and identifies the protocol– method groups satisfying the study-defined accuracymatching criterion at 50% sparsity. At 80% sparsity on Flowers-102, the LT baseline reaches 68.18% accuracy compared with 98.24% for the dense reference, marking a transfer recovery-stress point. The Flowers-102 protocol at 50% sparsity extends the nonCIFAR evidence beyond Imagenette. The LT, accuracy-only, BP-LT, and one-shot MP protocol–method groups of this protocol are matched under the study-defined accuracy rule. The dense ConvNeXt-Tiny reference reaches (98.51 ± 0.13)%, the LT baseline reaches (98.24 ± 0.46)%, and accuracyonly selection reaches 98.37%. Clean accuracy is now within 0.5 pp, while the behavioral gap remains. The LT baseline has −acc DBE = 0.739 and AcceptFlipτD = 7.28%, while BP-LT lowers the behavioral distance to 0.509 at 98.23% accuracy. One-shot MP fine-tuning has a lower behavioral distance of 0.388. One-shot MP remains within the same ±1.5 pp test recovery band and has AcceptFlipτD = 8.70%. Thus, a conventional pruning baseline can have a smaller aggregate behavioral distance while still changing the dense-threshold policy. Global random sparse and layerwise random sparse candidates reach only 57.56% and 51.26% accuracy. FGVC-Aircraft at 50% sparsity in Appendix Table 10 gives a second transfer protocol in which the LT, accuracy-only, BP-LT, and one-shot MP groups are matched under the studydefined accuracy rule. The dense ConvNeXt-Tiny reference

Method

Acc (%)

∆Acc (pp)

−acc DBE

OOD AUROC

Flip (%)

Flowers-102, 50% sparsity Dense

98.51

0.00

0.000

0.9815

0.00

LT Accuracy-only BP-LT One-shot MP

98.24 98.37 98.23 97.60

−0.27 −0.14 −0.28 −0.91

0.739 0.761 0.509 0.388

0.9831 0.9808 0.9855 0.9819

7.28 7.29 7.18 8.70

Random sparse Layerwise random

57.56 51.26

−40.95 −47.25

1.853 2.153

0.7009 0.7278

79.18 79.49

FGVC-Aircraft, 50% sparsity Dense

85.18

0.00

0.000

0.9557

0.00

LT Accuracy-only BP-LT One-shot MP

84.25 84.29 84.19 84.09

−0.93 −0.89 −0.99 −1.09

0.658 0.670 0.399 0.472

0.9426 0.9354 0.9400 0.9572

7.91 7.92 7.76 8.09

Random sparse Layerwise random

39.05 25.86

−46.12 −59.32

1.587 1.808

0.5472 0.5688

62.51 66.25

Table 10: Pretrained ConvNeXt-Tiny transfer evaluations with 224-pixel inputs. Sparse candidates are trained for 100 epochs, whereas one-shot MP follows a separate 20epoch fine-tuning schedule. In both 50% protocols, the LT, accuracy-only, BP-LT, and one-shot MP groups satisfy the study-defined accuracy-matching criterion; the randommask controls are recovery-stress rows. reaches (85.18 ± 0.18)%, the LT baseline reaches (84.25 ± 0.17)%, and accuracy-only selection reaches 84.29%. The sparse accuracy gap is about 1 pp. The LT baseline still differs from the dense reference’s behavior. The LT baseline has −acc DBE = 0.658, while BP-LT reduces the distance to 0.399 at 84.19% accuracy. Candidates with global random sparse and layerwise random sparse masks at the same sparsity reach only 39.05% and 25.86%, respectively, so this transfer result also distinguishes candidates trained with the LT mask from those trained with generic sparse masks at the same active parameter count.

Selection and Distance Robustness Selection-Source and Restricted-Pool Analysis Selection-source distribution. Across the 74 selection groups in the evaluation suite, accuracy-only selection chooses LT-baseline candidates in 35 groups, corresponding to 47.3%. BP-LT chooses LT-baseline candidates in 37 groups, corresponding to 50.0%. The remaining BP-LT selections come from LT reinitialization, global random sparse, layerwise random sparse, IMP, SNIP, or RigL candidates. These counts make the role of BP-LT explicit. In the allsparse analysis, BP-LT performs behavior-aware validation selection within the evaluated sparse pool, while the LT rows provide the direct replacement audit for LTs. Of the reported all-sparse BP-LT selections not drawn from LT-baseline candidates, 8 are LT-reinitialization candidates, 3 are RigL candidates, 1 is a SNIP candidate, and 25 are global random sparse or layerwise random sparse controls. Restricted-pool analysis. A restricted-pool analysis applies the validation-time selection rules to measured can-

didates after restricting the eligible pool to either the ticketfamily or LT-only pool. Under ticket-family eligibility, BPLT selects LT-baseline candidates in 52 of the 74 groups. Ticket-family BP-LT selects LT-reinitialization candidates in 19 groups, IMP candidates in two groups, and a SNIP candidate in one group. These results show that the same qualitative conclusion holds under ticket-family eligibility. For CIFAR-10 ResNet-18 at 50% sparsity, ticket-family −acc BP-LT lowers DBE from 0.648 to 0.594, compared with 0.446 for all-sparse BP-LT. At 80% sparsity, ticket-family BP-LT lowers the distance from 0.501 to 0.488, while allsparse BP-LT lowers the LT baseline’s distance to 0.437. In the larger CIFAR-10 pool at 50% sparsity, ticket-family BP-LT remains close to the all-sparse BP-LT result, with 0.854 compared with 0.832. In the large-pool protocol at 80% sparsity, ticket-family BP-LT lowers the LT baseline’s distance from 0.784 to 0.728, while all-sparse BP-LT lowers the LT baseline’s distance to 0.701. In the LT-only pool, the achievable improvement is determined by variation among measured LT candidates. The analysis is reselection among already measured models, which isolates eligibility effects within the fixed measured pool.

−acc DBE

Drank

DMah

50

LT Accuracy-only BP-LT

0.648 0.654 0.446

0.412 0.380 0.336

0.611 0.591 0.437

80

LT Accuracy-only BP-LT

0.501 0.890 0.437

0.379 0.432 0.403

0.413 0.725 0.314

90

LT Accuracy-only BP-LT

1.364 1.295 0.995

0.503 0.462 0.425

0.842 0.779 0.517

95

LT Accuracy-only BP-LT

1.059 1.099 1.014

0.383 0.386 0.371

0.743 0.781 0.666

Sparsity (%)

Method

Table 11: CIFAR-10 ResNet-18 distance sensitivity. are 1/2 J X 1 Drank (f, fD ) :=  [rj (f ) − rj (fD )]2  , J j=1 

and 

Distance Sensitivity Correlated coordinates can receive repeated weight in the diagonal summary; we therefore report coordinate-level signed results and evaluate rank- and covariance-aware distance variants here. The main distance is computed with diagonal standardization. Appendix Table 11 shows that the conclusion is stable under alternative scalings. BP-LT is either competitive with the best method or the best overall under both rank distance and shrinkage-based Mahalanobis distance across all four CIFAR-10 ResNet-18 sparsities. The absolute values change, as expected, because rank distance suppresses outliers and Mahalanobis distance accounts for cross-coordinate covariance. For both sensitivity analyses, we retain only those coordinates for which the dense-reference value is finite and at least one non-selection fitting row in the same protocol group has a finite value. Let J ≥ 1 denote the number of retained coordinates. For ranks and candidate-to-reference differences, missing evaluated-row values are imputed with the coordinate mean over all finite evaluated-row values in the group before forming the direction-signed vector u(f ) ∈ RJ . For the covariance fit, missing fitting-row values are instead imputed with the coordinate mean over the finite non-selection fitting-row values, giving ufit (g). These vectors are separate from the standardized coordinates Zj used in the main distance. Let rj (f ) be the percentile rank of uj (f ) among evaluated models in the same protocol group, and let the b LW , Ledoit–Wolf (LW) shrinkage covariance, denoted by Σ be estimated from the vectors ufit (g) associated with nonselection rows (Ledoit and Wolf 2004). Here, non-selection rows are the sensitivity-analysis fitting rows after excluding b † for the rows produced by validation selection. Write Σ LW b LW , namely the precision Moore–Penrose pseudoinverse of Σ matrix used by the implementation. The reported alternatives

1 b † δ(f ) DMah (f, fD ) := δ(f )⊤ Σ LW J δ(f ) := u(f ) − u(fD ).

1/2 ,

Additional Discussion Main Findings The experiments support four conclusions. First, clean-accuracy matching can coexist with behavioral mismatch and lower performance under deployment-relevant shift. For CIFAR-10 ResNet-18 at 50% sparsity, the LT baseline matches the dense reference’s accuracy to within −acc 0.01 pp but has DBE near 0.65. The same pattern persists at 80% and appears again in CIFAR-10 ResNet-34. The signed RobAcc comparisons in Table 1 and Appendix Tables 8 and 9 additionally show directional losses for accuracymatched LTs: 1.57 pp for CIFAR-10 ResNet-18 at 80%, 1.40 pp for CIFAR-100 ResNet-18 at 50%, 3.54 pp for CIFAR-10 WideResNet-28-2 at 80%, and 0.69 pp for Imagenette ViT-Tiny at 50%. The CIFAR-10 50% and Imagenette ConvNeXt-Tiny LTs match or exceed their dense references’ RobAcc values, showing that the direction varies by setting. Together, these results establish that clean recovery can conceal lower corruption performance and motivate signed, coordinate-level evaluation. Second, BP-LT turns the behavioral audit into a practical validation-time selection rule. BP-LT lowers mean behavioral distance relative to the LT baseline and accuracy-only selection across the primary CIFAR-10 multi-sparsity evaluation, the larger CIFAR-10 candidate-pool protocols, and the pretrained Imagenette study-band-matched protocols while maintaining the accuracy constraint. Sensitivity analyses using rank distance and shrinkage-based Mahalanobis distance keep the same qualitative picture. Cases where conventional pruning baselines are closer in behavior further establish the audit as method-agnostic. The result is a method-agnostic

operating-point compatibility audit paired with a validationtime sparse-candidate screening rule. Because BP-LT selects for behavioral closeness rather than directional utility, a −acc smaller DBE should not be read as an improvement in every coordinate; deployment-specific utility requirements still require signed metric checks or explicit performance floors. The coordinate-level results make this distinction concrete. At 50% and 80% sparsity for CIFAR-10 ResNet-18, −acc BP-LT lowers mean DBE from 0.648 to 0.446 and from 0.501 to 0.437, respectively, while the BP-LT RobAcc is lower than that of the corresponding LT baseline. In the −acc large-pool 80% protocol, BP-LT lowers DBE from 0.784 to 0.701, but the mean BP-LT AcceptFlipτD is 8.54%, slightly above the LT baseline’s 8.47%. Such outcomes are consistent with the objective: A reduction in the root-mean-square standardized distance can trade a larger deviation on one coordinate for smaller deviations on others, and validationtime selection need not preserve coordinate-wise ordering on the test distribution. Pareto dominance and worst-coordinate guarantees require additional constraints. Applications with asymmetric costs should combine the aggregate distance with coordinate-specific non-inferiority floors, applicationweighted objectives, or Pareto screening. The default all-sparse BP-LT analysis evaluates candidate screening within the measured pool. BP-LT selects an LTbaseline candidate in 37 of 74 selection groups; the remaining choices come from other eligible sparse sources. Ticketfamily and LT-only analyses isolate progressively stricter eligibility, but the gains attainable under those pools are limited by variation in the measured pool. Thus, the all-sparse results show that behavior-aware selection can find a closer sparse replacement; improving LT construction is a separate training objective. Third, the phenomenon is not explained by parameter count alone. The fixed-width small dense baselines are far from the dense reference in both accuracy and behavior, whereas sparse candidates with the same number of active parameters can achieve similar accuracy yet differ in behavior. Fourth, results from harder settings characterize the transition between sparse recovery and behavioral replacement. Results for CIFAR-100 at 50% sparsity support an accuracymatched behavioral-compatibility analysis. Results at 80% sparsity show the behavioral pattern as sparse recovery begins to degrade. The WideResNet-28-2 result adds a similar stress case. The LT baseline remains a strong classifier and stays within the test recovery band used in this paper, but the WideResNet accuracy gap is much closer to the tolerance edge than in the ResNet-18 settings at the 50% and 80% sparsity levels. Policy-Level Shift Interpretation Under this operatingpoint proxy, a flip rate of 8% means that reusing τD after sparse replacement routes about 8 of every 100 indistribution test inputs differently, which is approximately 0.08N out of N processed inputs: Automatic acceptance becomes review, or review becomes automatic acceptance. The first direction can add review-queue demand and latency; the second bypasses a review that the dense policy would

have required and can alter risk among automatically accepted inputs. Because AcceptFlipτD pools both directions, the acceptance-flip coordinate measures non-preservation of the routing policy rather than a net change in review rate or a signed utility change and is distinct from a top-1 label flip. The accompanying review-rate, automatically accepted error, and OOD acceptance coordinates help interpret the operational consequences. These experiments show routing non-preservation under the audited proxy policy. Translating the observed flips into reviewer outcomes, queueing cost, latency, or application-specific harm requires directionspecific flip rates and signed policy coordinates. Estimating a sparsity-specific causal effect additionally requires an independently retrained dense challenger to quantify excess churn beyond an ordinary model update. The present result establishes that accuracy recovery alone does not certify preservation of the audited action. The coexistence of these flips with small prediction-disagreement rates is the regime allowed by Theorem 1. Threshold crossings need not be label changes. Theorem 2 further gives the local first-order expansion under the stated path and density conditions. Theoretical Explanation of the Empirical Pattern Our theoretical results address two complementary questions: why the usual LT success criterion is insufficient for policycompatible replacement and how confidence shifts can change deployment decisions. Theorem 1 shows that even when every top-1 prediction agrees, reusing the dense reference’s fixed confidence threshold can change accept–review decisions for any fraction of inputs. Proposition 1 gives a finite-perturbation flip bound, while Theorem 2 makes the local policy-flip behavior quantitative. Along the stated local path, flip probability has a right-sided first-order expansion in the confidence perturbation, with κτD quantifying local sensitivity at the dense threshold. This local characterization complements the directly measured finite-model flip rates by explaining how confidence movement near the threshold can produce nonzero policy flips even when prediction disagreement is smaller. Together these results explain why clean-accuracy recovery alone leaves confidence-based deployment policies underdetermined. This theory establishes a limitation of the certificate rather than a sparsity-dependent lower bound on behavioral distance. The results are mask-agnostic and leave open whether a sufficiently expressive sparse model can preserve the measured behavior. The empirical variation across LT recipes, conventional pruning baselines, and sparsity levels, together with the gains from BP-LT, instead suggests that part of the observed gap is mitigable. The residual gap may reflect candidate-pool coverage, objectives that do not train for reference compatibility, validation noise, or capacity limits at a given sparsity. Three questions remain open: whether a jointly compatible sparse candidate exists, whether a training procedure can find such a candidate, and whether validation data can select such a candidate. Sparse Baselines and Selection Evidence The large-pool CIFAR-10 protocols strengthen the first two conclusions with a broader baseline set. IMP, SNIP, global random sparse, and layerwise random sparse candidates recover the dense

reference’s accuracy at 50% sparsity under the 200-epoch sparse training protocol. The separately scheduled one-shot MP baseline also recovers the reference’s accuracy, giving a stronger sparse-baseline comparison. At 80% sparsity, the LT baseline, LT reinitialization, IMP, SNIP, global random sparse, layerwise random sparse, and one-shot MP protocol–method groups again satisfy the test recovery band and thus provide study-band-matched evidence relative to a 94.53% dense reference. Even then, cleanaccuracy-excluded behavioral distances persist, and densethreshold AcceptFlipτD rates show that acceptance decisions continue to change. Under the 80% large-pool recipe, Staged MP falls in the recovery-stress stratum at 91.48%. Under the companion protocol with RigL, Staged MP, and a lower post-pruning fine-tuning learning rate, Staged MP recovers accuracy and becomes the sparse baseline closest in behavior to the dense reference among the LT baseline, RigL, one-shot MP, and −acc Staged MP. Even there, DBE remains 1.577 for Staged MP and 2.021 for RigL, so accuracy recovery still does not imply behavioral interchangeability. BP-LT selection reduces distance in the broader selectioneligible pool, with selected sources spanning several sparse methods. Together, these rows support two distinct conclusions: Behavior-aware sparse-candidate screening can identify a closer replacement within an evaluated pool, and LT baselines remain behaviorally distinguishable from dense references after accuracy recovery in the measured replacement regimes. Interpreting the Recovery Band The ResNet-34 result illustrates why accuracy gap and behavioral distance should be read together. Candidates with global random sparse masks have a low behavioral-compatibility distance in this setting and still satisfy the prespecified test recovery band. The accuracy of those candidates, however, is nearly 1 pp below the dense reference and lower than the accuracy of the LT baseline or one-shot MP alternatives. This lower-distance but −acc lower-accuracy pattern reflects our construction: DBE measures differences across behavioral coordinates, while clean aggregate accuracy is evaluated as a separate matching constraint. The explicit accuracy gap and band status therefore remain part of the replacement audit rather than footnotes to the behavioral distance. Pretrained and Transfer Settings The Imagenette protocols with 224-pixel inputs add evidence from pretrained architectures. With pretrained ConvNeXt-Tiny, the dense reference is highly accurate, and the LT baseline at 50% sparsity exceeds the reference’s clean accuracy, yet dense-threshold decisions, OOD behavior, PredDisagree, and aggregate behavioral distance still move. The pretrained ViT-Tiny protocol at 50% sparsity adds a transformer-backed case matched under the study-defined accuracy rule: The LT baseline recovers to within about 0.3 pp of the dense reference and still has an AcceptFlipτD rate of about 8%. The fine-grained transfer protocols extend the same pattern beyond the Imagenette protocols. The LT baseline for Flowers-102 at 50% sparsity recovers to within about 0.3 pp of the dense reference’s accuracy while still show-

−acc ing DBE = 0.739 and an AcceptFlipτD rate of about 7.3%. FGVC-Aircraft at 50% sparsity similarly recovers within about 1 pp and still shows a behavioral gap that behavior-aware selection reduces. These outcomes extend the behavioral-gap evidence from CIFAR and Imagenette to fine-grained transfer.

Related Work The lottery ticket hypothesis PASS framework (LTH-PASS) established a beyond-accuracy LT evaluation across shift, uncertainty, interpretability, and loss geometry (Chen et al. 2022b). Compression work also preserves per-example decisions or measures answer flips and output distances relative to a baseline model (Chee et al. 2022; Dutta et al. 2024). Building on these foundations, our distinction is the inherited operating point: We condition on clean-accuracy recovery, compare each challenger with the corresponding incumbent, and audit the binary routing action produced by applying the same fixed dense-derived threshold to both models. This view connects sparse replacement to backward-compatible model updates. Prior work studies prediction churn, negative flips, downstream pipeline failures, and compatibility-aware objectives or selection (Fard et al. 2016; Bansal et al. 2019; Srivastava et al. 2020; Yan et al. 2021; Ötles, Denton, and Wiens 2023). Our policy coordinate applies the paired comparison principle to accept–review routing rather than only predicted labels or correctness. LT recovery. The LT hypothesis showed that dense networks can contain sparse subnetworks that, when initialized at their original weights and trained in isolation, recover test accuracy comparable to that of their dense counterparts (Frankle and Carbin 2019). Subsequent work clarified the roles of rewinding, late-training dynamics, and pruning schedules (Frankle et al. 2019, 2020; Renda, Frankle, and Carbin 2020), while large-scale studies showed that sparse recovery depends strongly on architecture, optimizer, and training recipe (Gale, Elsen, and Hooker 2019). Work on LT transfer further showed that sparse initializations can generalize across related datasets and optimizers (Morcos et al. 2019). Beyond-accuracy LT evaluation extends this line of work beyond clean-accuracy recovery. Most directly, LTHPASS asks whether sparse subnetworks can replace dense counterparts after accuracy recovery and evaluates distribution shift, uncertainty, interpretability, and loss geometry (Chen et al. 2022b). We build on this foundation by testing compatibility with an inherited incumbent operating point and distinguishing symmetric continuity from signed service-level preservation. Pruning and sparse training baselines. LT methods build on classical and modern pruning methods that remove weights according to saliency or magnitude and retrain the surviving network (LeCun, Denker, and Solla 1989; Han et al. 2015; Han, Mao, and Dally 2016; Zhu and Gupta 2018). Broader surveys and controlled retraining studies caution that compression claims are recipe-dependent (Liu et al. 2019; Blalock et al. 2020). Generalization analyses likewise show that pruning effects cannot be reduced to parameter count alone (Jin et al. 2022). Pruning at initialization and dynamic

sparse training provide competitive sparse alternatives, including SNIP, GraSP, SynFlow, and RigL (Lee, Ajanthan, and Torr 2019; Wang, Zhang, and Grosse 2020; Tanaka et al. 2020; Evci et al. 2020). Audits of pruning at initialization further show that layerwise sparsity allocation explains much of the advantage over random pruning (Frankle et al. 2021). We therefore evaluate LT baselines together with controls based on conventional pruning, pruning at initialization, dynamic sparse training, and random sparsity rather than treating any single sparse construction as definitive. Reliability after compression. Compression can change which examples a network forgets, which groups suffer, and how robust the model is under shift (Hooker et al. 2019; Liebenwein et al. 2021; Tran et al. 2022). Related sparsereliability work studies calibrated or OOD-robust LTs and sparse training (Venkatesh et al. 2020; Diffenderfer et al. 2021; Lei et al. 2023). Privacy-risk diagnostics based on an MIA provide another clean-accuracy-excluded behavioral coordinate for comparing trained models (Shokri et al. 2017; Yeom et al. 2018; Carlini et al. 2022). Taken together, prior work already shows that compression can alter examplelevel outputs and reliability even when aggregate accuracy is similar, with effects that may be adverse or beneficial depending on the diagnostic and recipe (Hooker et al. 2019; Liebenwein et al. 2021; Diffenderfer et al. 2021; Tran et al. 2022; Chen et al. 2022b; Chee et al. 2022; Dutta et al. 2024; Tong et al. 2026). Post-hoc pruning benchmarks additionally expose calibration–corruption trade-offs whose directions can depend on architecture and pruning recipe (Mitra, Schwalbe, and Klein 2024). The closest compression precedents ask for broad capability parity, preserve per-example decisions during compression, or measure answer flips and output distances relative to a baseline (Chen et al. 2022b; Chee et al. 2022; Dutta et al. 2024). Our contribution centers on the downstream object held fixed: We condition on cleanaccuracy recovery, measure paired deviation from a matched dense reference, and test the binary routing action induced by reusing the same dense-derived proxy threshold. This design yields an operating-point compatibility audit and a validation-time screening rule for strict drop-in replacement. Calibration, OOD, and selective policies. Calibration and uncertainty studies show that predictive confidence can be misleading even when in-distribution accuracy is high (Naeini, Cooper, and Hauskrecht 2015; Guo et al. 2017; Snoek et al. 2019). OOD detection methods and common-corruption benchmarks provide operational probes of distribution-shift behavior (Hendrycks and Gimpel 2017; Hendrycks and Dietterich 2019). Selective classification studies abstention and review policies rather than forced prediction (Geifman and El-Yaniv 2017). These literatures motivate our coordinates for calibration, OOD behavior, corruption accuracy, and dense-threshold policies, which test preservation of the dense reference’s operating point after sparse replacement. Related model-update work measures prediction churn as paired, example-level label disagreement across model versions, including settings in which aggregate accuracy remains similar (Fard et al. 2016; Bahri and Jiang

2021). Backward-compatibility work further studies new errors, downstream pipeline failures, human expectations, threshold-dependent compatibility, and compatibility-aware objectives or selection (Bansal et al. 2019; Srivastava et al. 2020; Yan et al. 2021; Ötles, Denton, and Wiens 2023). Our AcceptFlipτD coordinate uses the same paired-comparison principle for a different object: the binary accept–review action induced by applying the fixed dense threshold to both models. The acceptance-flip coordinate symmetrically pools both flip directions under the fixed threshold and does not require test labels; unlike prediction churn or a negative flip, the policy coordinate measures preservation of a proxy routing policy rather than a change in predicted class or correctness. Representations and multiplicity. Similarity methods such as CKA show that models with comparable accuracy need not compute the same internal features (Kornblith et al. 2019). More broadly, the Rashomon effect view, predictive multiplicity, and underspecification show that highperforming models can make different individual decisions or encode different solutions (Breiman 2001; Marx, Calmon, and Ustun 2020; D’Amour et al. 2022). We use representation distance and prediction disagreement as behavioral probes alongside calibration, OOD response, dense-threshold policy decisions, privacy-risk diagnostics, and corruption accuracy. Representation distance is a compatibility diagnostic when embeddings or intermediate features are reused by downstream heads, retrieval, or monitoring; the representation diagnostic is not a necessary component of a label-only replacement contract.

Limitations and Future Extensions Scope and attribution. We study unstructured sparse replacements in controlled vision tasks. Extending the audit to ImageNet-scale and language models, structured sparsity, and end-to-end latency and energy remains future work; active parameter counts alone do not establish hardware speedups. We use the incumbent as the sole dense reference, with no independently retrained dense challenger. The results therefore show that clean-accuracy recovery does not certify continuity for the evaluated sparse models, but do not isolate excess incompatibility caused by sparsity from ordinary retraining churn (Fard et al. 2016; Bahri and Jiang 2021; Bansal et al. 2019). Dense-to-dense controls are needed for that causal comparison. −acc Audit, selection, and theory. DBE summarizes only the active, protocol-specific coordinates and is neither exhaustive nor a signed utility measure. AcceptFlipτD audits one densederived threshold and pools both routing directions. A production audit should predeclare required coordinates, margins, missingness rules, and uncertainty or non-inferiority checks, adding threshold sweeps or directional costs when relevant. BP-LT selects from a finite measured pool and neither guarantees nor constructs a jointly compatible candidate. Broader pools, reference- or threshold-aware training, and constrained or Pareto selection are natural extensions. The theory establishes non-identifiability and a local boundary mechanism; linking its coefficient to finite trained models remains open.