Don't Replace, Complement: A Detector-Agnostic Subspace Signal for Near-OOD Detection
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
Don’t Replace, Complement: A Detector-Agnostic Subspace Signal for Near-OOD Detection Wonseok Shin Songkuk Kim* Yonsei University, Republic of Korea
arXiv:2611.06688v1 [cs.LG] 4 Nov 2026
{wonseok.shin,songkuk}@yonsei.ac.kr
Abstract
distribution (OOD) inputs to ensure safety and trustworthiness. Compared to alternative paradigms, post-hoc OOD detection is particularly well-suited for real-world deployment, as it operates directly on pre-trained classifiers without requiring model retraining, architectural modifications, or exposure to auxiliary OOD data. While these posthoc methods are now highly effective at detecting farOOD inputs—samples semantically distant from the training data—they exhibit degraded performance on near-OOD inputs, which are visually and semantically proximate to the in-distribution (ID) classes. This performance gap is a structural consequence of how features are activated. Far-OOD inputs activate fewer of their class-specific feature directions, making them easily distinguishable via global feature summaries such as density, residual energy, or classifier confidence. In contrast, near-OOD inputs lie directly on the in-distribution manifold and activate class-specific features in a manner closely resembling ID inputs. As a result, the standard global feature axes that successfully isolate far-OOD data leave nearOOD inputs almost indistinguishable from ID data, as they inherently fail to capture the fine-grained internal structure of individual classes. To overcome this structural limitation, we propose a different approach. Using a sparse autoencoder (SAE), we identify the feature neurons most characteristic of each class and treat their corresponding directions as the axes of a compact, per-class subspace. An input is then evaluated based on its projection into the subspace of its predicted class: a genuine ID sample naturally aligns with the dense regions of the training data, whereas a near-OOD sample exhibits measurable deviation within this subspace. By explicitly capturing this within-class structure, we derive a signal z that is informative precisely where global scores lack discriminative power, providing an effective mechanism for near-OOD detection. A further challenge is that relying on a single standalone score typically forces an inherent trade-off between near- and far-OOD detection. Rather than attempting to resolve this tension within yet another standalone detector,
Post-hoc out-of-distribution (OOD) detection is mature on far-OOD yet remains weak on near-OOD. These two settings differ in their use of class-characteristic feature directions: far-OOD inputs largely fail to activate those of their predicted class, whereas near-OOD inputs activate them much as in-distribution samples do. Because the axis separating far-OOD leaves near-OOD almost indistinguishable from ID, a score optimized for one setting is misaligned with the other; where a detector already saturates far-OOD, added near-OOD sensitivity is bought partly against it. This divergence motivates a targeted near-OOD signal designed to complement existing scores. We propose PLUS (Per-class Low-dimensional sUbspace Signal). For each class, PLUS selects the m most class-active neurons from a sparse autoencoder, orthonormalizes the corresponding decoder directions to form a low-dimensional subspace, and scores an input by its per-class whitened squared Mahalanobis distance z within the subspace of its predicted class. Designed as a dedicated complement rather than a standalone detector, the signal z is fused additively with any calibrated base score at a single, fixed weight. This weight is a single global constant: no value is adjusted per backbone, per OOD dataset, or per deployment, and all calibration statistics are estimated from ID validation data alone. When applied to 18 existing detectors spanning diverse method families, z consistently improves near-OOD AUROC in all 126 detector–backbone combinations. Far-OOD performance is broadly maintained or improved, with only marginal declines where the base score is near-optimal, making z a robust, detector-agnostic complement across backbones.
1. Introduction Deploying machine learning models in open-world environments requires the reliable identification of out-of* Corresponding author.
1
(a) Training ID-train only
Classifier f
ID-train dataset
Sparse Autoencoder
Fixed Support Sc top-m class-active neurons
(b) Inference Test samples
Class Statistics
Orthonormal Basis Qc
(µc , Σc )
(Qĉ , µĉ , Σĉ ) ĉ
Predicted Class ĉ
PLUS Score 2 Qĉ⊤ (x − b) − µĉ 2
z(x)
cz (x)
Standardize
−1/2
z(x) = Σĉ
×β
Input image
Classifier f
x
⊕
feature
Base Detector A Score (MSP, Energy, MDS++, . . . )
sA (x)
Standardize
Final Score S(x) = cA (x) + β cz (x)
cA (x)
Figure 1. Overview of PLUS. (a) Training: Using only ID training data, a TopK-SAE is trained on backbone features. For each class c, a fixed support Sc of its m most active neurons is identified, orthonormalized into a basis Qc , and characterized by subspace statistics (µc , Σc ). (b) Inference: Given an input image, the classifier produces the ℓ2 -normalized feature x and the predicted class ĉ. PLUS then computes the whitened deviation z(x) within the predicted class’s subspace. Both the base detector score and z(x) are standardized using correctly classified ID validation negatives and additively fused with a single fixed weight β = 0.5.
we introduce PLUS (Per-class Low-dimensional sUbspace Signal). PLUS is designed as a complement that can be seamlessly attached to any existing post-hoc detector, additively integrating our near-OOD signal z via a single, fixed weight. In doing so, PLUS preserves the strong far-OOD capabilities inherent to the base detector while markedly boosting its near-OOD sensitivity—all without network retraining or architectural modifications, and with every hyperparameter held as a single global constant and all calibration statistics estimated from ID validation data alone. Our primary contributions are as follows:
randomly chosen SAE neurons yield essentially no nearOOD gain, and across all seven backbones, the SAE subspace is more consistent than per-class PCA, with a smaller worst-case shortfall (Section 6.1, Appendix D). 3. A Detector-Agnostic Complementary Protocol. Fused additively via a single fixed weight, z improves the near-OOD AUROC of all 18 evaluated detectors across seven diverse architectures while broadly maintaining far-OOD performance (Section 5.1, Appendix C). This improvement transcends a generic ensemble effect: incorporating z into any of the 714 evaluated two-detector ensembles consistently yields further improvements in near-OOD AUROC across all cases (Section 6.4). Furthermore, all hyperparameters are defined as single global constants shared across backbones and datasets, with calibration frozen on ID validation negatives.
1. Attributing the Near-OOD Gain to Subspace and Metric. A subspace×metric ablation on the three main backbones (Section 6.1) reveals that the two components of z play distinct roles. The subspace is essential: applying the same per-class metric within a random 16-dimensional subspace yields no near-OOD gain over MDS++. The per-class metric is required to complete this gain: within our subspace, a shared or isotropic metric recovers only part of the improvement, and a shared metric recovers almost none on ResNet-50. The nearOOD signal thus requires modeling each class’s own anisotropy along its most active directions.
2. Related Work Post-hoc OOD Detectors. Post-hoc OOD detection typically leverages representations from a frozen classifier without requiring model retraining. Initial approaches primarily focus on the output-space confidence. Methods such as MSP [13], MaxLogit [14], and Energy [20] utilize the maximum softmax probability, raw logits, or free energy to flag inputs when the network is unconfident. Extensions like GEN [21] and KL-Matching [14] further refine this by measuring entropy or comparing posteriors against classconditional means. To overcome the inherent overconfidence of softmax outputs, subsequent research has shifted toward featurespace distance and density estimation. Methods such as
2. A Structural Use of a Sparse Autoencoder for NearOOD. PLUS uses an SAE structurally rather than as a feature score: each class’s most active neurons define a fixed, low-dimensional subspace in which z is measured (Section 3). Fixing the support is essential, because raw SAE activations change support from image to image and provide no consistent per-class coordinate system (Section 3.2). The choice of neurons also matters: 16 2
Mahalanobis [18] and RMDS [31] fit class-conditional Gaussians to score samples based on their deviation from typical ID feature distributions. Recent variants, including MDS++/RMDS++ [25] and MahaVar [16], enhance stability by normalizing features or incorporating distance-variance terms. Non-parametric approaches such as KNN [34], SHE [41], and NNGuide [27] similarly identify OOD inputs by measuring their deviation from stored ID prototypes or neighborhood densities. Another prominent line of work examines the geometric structure of the feature space. Building upon NuSA [5], ViM [39] scores the residual norm of features outside the principal ID subspace. Motivated by neural collapse, NECO [1] instead scores the relative norm within this subspace. Kernel PCA [10] extends ViM’s reconstruction-error view to a non-linear, kernel-induced subspace. Other detectors rely on different cues: fDBD [19] measures the distance to decision boundaries, and GradNorm [15] scores the norm of the gradient. Complementary to these scoring functions, shaping techniques such as ReAct [33], ASH [8], and DICE [32] truncate, rescale, or sparsify activations or weights to amplify the margin between ID and OOD samples. While highly effective, the operative signal across all these families is inherently global: they measure general atypicality, distance to the overall ID feature cloud, or deviation from a class-agnostic subspace shared by all classes (e.g., ViM [39] or Kernel PCA [10]). Consequently, they are structurally blind to near-OOD inputs, which lie on the ID manifold and mimic the typical features of their predicted class. Our method is designed specifically to address this gap. Rather than proposing another standalone detector, PLUS introduces a targeted, per-class deviation signal designed to be additively fused with any of the existing global scores mentioned above.
In the context of OOD detection, Oh et al. [26] recently utilized SAEs by scoring samples based directly on the activation typicality of their latent neurons. PLUS, however, adopts a fundamentally different perspective. We make no claims of human-level semantic interpretability; instead, PLUS treats the SAE purely as a geometric tool. We identify the neurons most characteristically active for each class and use their corresponding decoder weights to construct a fixed, low-dimensional projection basis. By exploiting the overcomplete dictionary as a source of class-specific coordinate directions, PLUS bypasses the often-debated pursuit of semantic interpretability, leveraging the SAE for robust structural isolation.
3. Method Our method, PLUS, constructs a low-dimensional per-class projection basis from the decoder of a sparse autoencoder (SAE). At inference time, it produces a single scalar z(x) representing the whitened deviation of an input within the subspace of its predicted class. This signal is then fused additively with any calibrated base detector score at a single fixed weight. Figure 1 provides an overview of PLUS.
3.1. Preliminaries Let f denote a frozen, pretrained classifier and let x′ ∈ Rd be its penultimate-layer feature. To remove magnitude bias, we operate on the ℓ2 -normalized feature x = x′ /∥x′ ∥2 . The classifier’s prediction ĉ(x) ∈ {1, . . . , C} is used directly. To construct our per-class subspaces, we extract an overcomplete dictionary of p ≫ d unit-norm directions (referred to as neurons) from the normalized ID-train features x using a TopK sparse autoencoder (SAE) [11]. The SAE encodes the input into a sparse activation vector with at most k non-zero entries: h(x) = \mathrm {ReLU}\big (\mathrm {TopK}_k\big (W_{\text {enc}}(x - b) + b_{\text {enc}}\big )\big ) \in \mathbb {R}_{\geq 0}^{p}. (1)
Sparse Autoencoders. Sparse Autoencoders (SAEs) recently gained prominence in language-model interpretability as a dictionary learning tool. They employ an overcomplete, sparsely activated architecture to decompose dense representations into monosemantic (interpretable) feature directions [3, 6]. Beyond the standard ReLU architecture, modern variants like TopK [11], Gated [29], and JumpReLU [30] SAEs balance reconstruction fidelity and sparsity differently, enabling dictionaries to scale up to millions of neurons [35]. However, whether SAEs genuinely disentangle humaninterpretable concepts remains widely debated. Empirical evaluations indicate that SAE features only partially align with true causal directions [24], and models trained with different random seeds often yield substantially different dictionaries [28]. This lack of stable semantic alignment is even more pronounced in computer vision, where the evidence for human-level interpretability is still preliminary.
It then reconstructs the feature via a linear decoder x̂ = Wdec h(x) + b, where Wdec = [w1 , . . . , wp ] ∈ Rd×p forms the dictionary. Trained unsupervised via standard reconstruction and auxiliary losses, the SAE provides the IDtrain activations h(x) and the decoder dictionary {wj }pj=1 needed to establish our per-class supports.
3.2. Subspace Construction Raw TopK-SAE latents activate a different support on every image; within a single class, the top-k active-neuron sets of two images share only a small fraction of neurons (intra-class support Jaccard ≈ 0.09 on ViT-B/16, versus 1 by construction for the fixed support). A per-image support implies a shifting coordinate system, rendering any class-conditional distribution undefined. To make the geometric object “distribution of class c in its subspace” wellposed, we must establish a consistent projection frame. 3
3.4. Calibration and Fusion
For each class c, we fix the support Sc ⊂ {1, . . . , p} (|Sc | = m = 16) P as the indices of the m neurons maximizing h̄j (c) = n1c i:yi =c hj (xi ), where nc is the number of ID-train samples for class c. Computed over post-TopK activations, this selection favors the neurons that fire most frequently and strongly for class c. To build a well-defined orthonormal basis for this support, we collect the corresponding decoder columns Wdec [: , Sc ] ∈ Rd×m and apply a thin QR factorization with a fixed sign convention: \begin {aligned} W_{\text {dec}}[:, S_c] &= Q_c R_c, \quad Q_c \in \mathbb {R}^{d \times m}, \\ Q_c^\top Q_c &= I_m, \quad \text {diag}(R_c) > 0. \end {aligned}
Let sA (x) denote the score of any base detector A, with its sign oriented so that larger indicates more OOD (for detectors with the opposite native convention, e.g., MSP, we use −sA ; the orientation is a fixed property of each method). Because sA and z live on unrelated scales, both are standardized on the correctly-classified ID-validation negatives using a single global mean and standard deviation per channel, frozen once: c_A(x) = \frac {s_A(x) - \nu _A}{\tau _A}, \quad c_z(x) = \frac {z(x) - \nu _z}{\tau _z},
where (ν• , τ• ) are the mean and standard deviation of the corresponding score over these negatives, computed once and frozen. Standardization is a monotone affine map, leaving each channel’s individual AUROC unchanged; its role is to express both channels in common ID-standard-deviation units, so that the fixed weight below is interpretable as a channel ratio. The deployed score combines these complementary mechanisms via:
Qc spans the same subspace as the selected neurons but in an orthonormal frame, ensuring that correlations among dictionary directions do not distort the distance metric. With the sign convention diag(Rc ) > 0, the factorization is unique, making Qc a purely deterministic function of Sc and the shared dictionary.
3.3. The Signal z Given the fixed orthonormal basis Qc ∈ Rd×m for each class c, the empirical mean and covariance of the centered, projected ID-train features in the m-dimensional subspace are given by:
S(x) = c_A(x) + \beta c_z(x), \quad \beta = 0.5.
(6)
We fix β = 0.5 as an a priori base-favored choice (a 2:1 ratio reflecting higher trust in the established base detector), which is neither fitted, tuned, nor selected by any algorithmic rule on any split. We emphasize that β = 0.5 is a deployment default, not a claim of optimality. The sensitivity analysis (Section 6.2) shows the trade-off is backbonedependent: near-OOD gains saturate around β ≈ 0.5 on well-conditioned backbones but continue rising beyond it on others, while smaller weights (e.g., β = 0.25) retain most of the near-OOD gain at a lower far-OOD cost. A practitioner with deployment-specific priorities, or validation data of their own, may therefore prefer a different operating point on this curve; all results reported in this paper use the single fixed β = 0.5.
\begin {gathered} \mu _c = \frac {1}{n_c} \sum _{i : y_i = c} Q_c^\top (x_i - b) \in \mathbb {R}^m, \\ \Sigma _c = \frac {1}{n_c} \sum _{i : y_i = c} u_{i,c} u_{i,c}^\top + \epsilon I_m, \end {gathered} (3)
d where ui,c = Q⊤ c (xi − b) − µc , b ∈ R is the SAE decoder −3 bias, and ϵ = 10 is a ridge regularization parameter fixed across all classes and backbones. The low dimensionality (m = 16) ensures that Σc is well-conditioned, avoiding the singularity inherent to full d-dimensional per-class covariances at ImageNet scales. At test time, given a normalized feature x and predicted class ĉ = ĉ(x), the centered feature is projected onto Qĉ to evaluate the whitened squared Mahalanobis distance within the corresponding subspace:
z(x) = \big \lVert \Sigma _{\hat {c}}^{-1/2} \big ( Q_{\hat {c}}^\top (x - b) - \mu _{\hat {c}} \big ) \big \rVert _2^2.
(5)
(2)
4. Experimental Setup 4.1. Benchmark, Backbones, and Baselines Benchmark. We evaluate on the OpenOOD v1.5 framework [42] using ImageNet-1K [7] as the in-distribution (ID) dataset. Near-OOD evaluation comprises SSB-hard [38] and NINCO [2], while far-OOD evaluation covers iNaturalist [37], Textures [4], and OpenImage-O [39]. Performance is evaluated using AUROC and FPR95.
(4)
Here, Qĉ spans the directions in which class ĉ concentrates −1/2 its activations, while Σĉ weights each deviation by that class’s own variability, amplifying displacements along directions where the class varies little, and thereby quantifies atypicality along the coordinates most relevant to the predicted class. z discards the component of x orthogonal to the subspace, measuring instead how atypical the insubspace coordinates are relative to the class’s own ID distribution.
Backbones. We evaluate the three standard OpenOOD backbones: ViT-B/16 [9], ResNet-50 [12], and Swin-T [22]. In addition, to assess generalization across diverse architectural families, we evaluate four further ImageNet backbones (seven in total): ResNet-18 [12], Wide-ResNet50-2 [40], 4
ViT-B/16 Method
Near-OOD
Far-OOD
ResNet-50 Overall
Near-OOD
Far-OOD
Swin-T Overall
Near-OOD
Far-OOD
Overall
AUROC ↑ FPR95 ↓ AUROC ↑ FPR95 ↓ AUROC ↑ FPR95 ↓ AUROC ↑ FPR95 ↓ AUROC ↑ FPR95 ↓ AUROC ↑ FPR95 ↓ AUROC ↑ FPR95 ↓ AUROC ↑ FPR95 ↓ AUROC ↑ FPR95 ↓ MSP MSP + PLUS MaxLogit MaxLogit + PLUS Energy Energy + PLUS GEN GEN + PLUS KL-Matching KL-Matching + PLUS ReAct ReAct + PLUS ASH ASH + PLUS DICE DICE + PLUS ViM ViM + PLUS KNN KNN + PLUS MDS MDS + PLUS RMDS RMDS + PLUS MDS++ MDS++ + PLUS RMDS++ RMDS++ + PLUS GradNorm GradNorm + PLUS SHE SHE + PLUS NNGuide NNGuide + PLUS fDBD fDBD + PLUS
73.52 78.13 68.30 75.65 62.41 73.27 76.30 79.68 74.41 77.04 69.26 76.15 53.21 70.51 65.42 74.63 77.03 79.93 73.93 78.58 79.03 80.94 79.99 81.47 79.87 81.09 80.07 81.21 39.28 59.37 76.02 79.43 57.29 70.16 73.27 78.01
81.85 68.82 92.25 84.39 93.19 89.51 70.78 65.87 77.25 70.52 84.49 74.70 94.43 88.88 85.32 72.95 73.63 67.98 70.67 67.14 66.11 63.82 65.73 63.17 65.63 63.87 65.84 63.54 94.71 92.45 71.08 66.47 91.91 85.64 73.67 69.29
86.04 88.44 83.54 87.44 78.98 85.86 91.35 92.13 87.70 88.92 85.69 89.24 51.56 72.17 82.57 88.27 92.93 93.15 90.71 91.72 92.65 92.54 92.61 92.36 93.29 92.26 92.64 91.69 41.75 65.04 92.38 92.72 74.29 84.62 89.36 90.84
51.69 43.86 79.23 56.73 85.35 69.29 32.23 30.53 48.45 40.57 53.93 42.91 96.77 89.91 51.31 38.21 29.15 26.30 32.28 28.36 29.94 28.09 28.79 27.86 24.93 26.58 27.02 28.30 92.65 90.95 27.53 26.17 74.46 57.68 38.99 33.80
81.03 84.32 77.45 82.73 72.35 80.82 85.33 87.15 82.39 84.17 79.12 84.00 52.22 71.51 75.71 82.81 86.57 87.87 84.00 86.46 87.20 87.90 87.56 88.00 87.92 87.79 87.61 87.50 40.76 62.77 85.83 87.41 67.49 78.84 82.93 85.71
63.75 53.84 84.44 67.80 88.48 77.38 47.65 44.66 59.97 52.55 66.15 55.63 95.84 89.49 64.91 52.10 46.94 42.97 47.64 43.87 44.41 42.38 43.57 41.98 41.21 41.50 42.55 42.40 93.47 91.55 44.95 42.29 81.44 68.87 52.87 48.00
76.02 79.30 76.46 80.86 75.89 80.71 76.85 81.11 76.63 78.51 77.38 80.87 78.17 81.34 73.08 79.37 72.04 77.37 68.56 75.31 55.44 68.50 76.71 80.50 74.71 78.23 75.75 79.90 72.96 78.90 73.70 79.64 77.53 81.10 76.63 80.01
65.68 63.38 67.85 63.19 68.56 63.57 65.29 62.18 72.53 65.81 66.74 63.27 63.33 61.32 72.44 67.62 71.30 67.56 74.24 70.51 85.40 77.73 65.33 62.19 69.05 66.51 69.41 65.22 78.88 71.36 73.13 66.73 65.92 62.60 64.69 62.70
85.23 89.64 89.58 92.98 89.47 92.95 89.77 92.91 87.59 89.40 93.67 94.09 95.74 95.61 90.95 93.33 92.61 94.80 90.30 94.14 74.10 87.46 86.17 91.67 95.13 95.45 85.22 90.88 90.25 92.94 90.85 93.67 94.59 94.83 92.78 94.41
51.45 40.88 38.20 29.22 38.39 29.01 35.62 28.17 46.81 37.95 26.31 24.35 19.52 18.64 41.81 30.61 24.86 21.34 34.25 24.54 62.97 44.55 41.77 32.30 20.37 19.56 51.98 35.71 47.91 35.59 41.90 30.16 25.20 22.02 26.54 23.32
81.55 85.50 84.33 88.13 84.04 88.05 84.60 88.19 83.21 85.04 87.15 88.80 88.71 89.90 83.80 87.75 84.38 87.83 81.61 86.61 66.64 79.88 82.39 87.20 86.96 88.56 81.43 86.49 83.34 87.33 83.99 88.06 87.76 89.34 86.32 88.65
57.14 49.88 50.06 42.81 50.46 42.83 47.49 41.78 57.10 49.09 42.48 39.92 37.05 35.71 54.06 45.42 43.44 39.82 50.25 42.93 71.94 57.82 51.19 44.26 39.85 38.34 58.95 47.52 60.30 49.90 54.39 44.79 41.49 38.25 41.80 39.07
76.75 80.02 75.66 79.79 73.23 78.95 78.97 81.59 76.11 78.28 75.64 79.82 46.47 70.79 48.46 64.43 75.31 78.92 71.40 77.53 75.20 78.70 78.22 80.69 78.92 80.86 79.70 81.37 47.58 64.39 76.48 80.23 60.79 71.58 73.02 77.89
71.06 64.33 80.76 72.02 83.31 75.61 63.62 60.81 75.46 67.89 72.66 66.07 95.04 87.38 96.87 89.72 74.67 67.34 72.01 67.83 68.81 65.27 66.70 63.77 67.66 65.00 66.61 63.97 93.82 91.83 76.57 69.76 91.74 84.70 72.81 67.66
86.30 88.55 84.80 87.79 81.32 86.26 90.98 91.79 88.32 89.43 88.23 90.21 44.64 74.64 60.68 76.98 93.26 93.95 89.24 91.62 91.51 92.55 92.27 92.74 93.67 93.00 92.44 91.81 35.47 58.65 89.62 91.20 81.23 88.04 89.58 91.21
49.16 42.96 67.81 51.50 75.59 58.87 32.59 30.42 43.33 38.52 43.01 35.96 94.71 81.65 94.13 82.35 24.52 24.04 34.67 29.37 29.97 28.17 28.93 27.73 25.17 25.24 27.68 27.90 96.61 94.52 47.11 36.18 65.37 49.94 34.88 31.41
82.48 85.14 81.14 84.59 78.08 83.34 86.17 87.71 83.44 84.97 83.19 86.05 45.37 73.10 55.79 71.96 86.08 87.94 82.10 85.98 84.98 87.01 86.65 87.92 87.77 88.14 87.34 87.63 40.32 60.95 84.36 86.81 73.05 81.45 82.96 85.88
57.92 51.51 72.99 59.70 78.68 65.57 45.00 42.57 56.18 50.27 54.87 48.01 94.84 83.94 95.23 85.30 44.58 41.36 49.61 44.76 45.51 43.01 44.04 42.14 42.16 41.15 43.25 42.33 95.50 93.44 58.89 49.61 75.92 63.84 50.05 45.91
Table 1. PLUS as a detector-agnostic near-OOD complement. For each of 18 post-hoc baselines on the three standard OpenOOD backbones, the standalone score (plain row) is followed by the β=0.5 fusion (shaded “base + PLUS” row). Improvements are shown in bold; an underline marks the few metrics where the fusion slightly regresses the standalone (AUROC higher / FPR95 lower is better).
ConvNeXt-B [23], and MaxViT-T [36]. Detailed checkpoint sources for all backbones are provided in Appendix A.
of neurons to p = 6144 across all backbones, while specific training procedures follow the original method of [11].
Baselines. We compare against a comprehensive suite of post-hoc detectors from OpenOOD v1.5 as well as recent state-of-the-art methods: MSP [13], MaxLogit [14], Energy [20], GEN [21], KL-Matching [14], ReAct [33], ASH [8], DICE [32], ViM [39], KNN [34], MDS [18], RMDS [31], MDS++/RMDS++ [25], GradNorm [15], SHE [41], NNGuide [27], and fDBD [19]. Our implementations are based on the official OpenOOD v1.5 codebase, while detectors not natively supported by the framework follow the official repositories released by their respective authors.
Neuron Demand Across Backbones. The near-OOD performance response to the neuron count p is largely flat. Sweeping p from 2× to 12× the feature dimension alters the z-standalone near-OOD AUROC by at most 0.5 points on four of the seven backbones, with the response itself exhibiting noise with respect to p. The two backbones that do demonstrate a clear preference diverge in opposite directions: ResNet-18 steadily improves as neurons are added, whereas MaxViT-T peaks at the minimal overcomplete capacity, even though both share the exact same feature dimension (d = 512) and expansion ratio. We therefore adopt a single fixed p = 6144 across all models rather than tuning it per backbone. As detailed in Appendix B, this single value costs at most ≈ 1 point of z-standalone near-OOD AUROC relative to the best per-backbone p up to 12×, and this gap shrinks further once z is fused with a base detector.
4.2. Sparse Autoencoder Training and Hyperparameters. We employ a TopK sparse autoencoder trained on ℓ2 -normalized ID-train penultimate features. For each input, the encoder retains only the k = 32 largest pre-activations while zeroing the rest, with an AuxK loss reviving dead units to keep the dead-neuron rate negligible. We uniformly fix the number
Computational cost. PLUS leaves the base classifier frozen and requires only a one-time SAE fitting on the ID5
train features. Training the SAE on cached Swin-T features takes 7–9 minutes on a single NVIDIA RTX A5000, peaking at 3.7 GiB of memory. At inference, computing z requires one d×16 projection and one 16×16 whitening operation per image. This takes 0.4–0.7 µs, representing at most 0.2% of a typical backbone forward pass. Storing the per−1/2 class statistics {Qc , µc , Σc } for 1,000 classes requires 32–126 MiB in fp32, depending on d.
Near-OOD
Backbone
Far-OOD
SSB-hard NINCO iNaturalist Textures OpenImage-O ResNet-18
18/0
18/0
17/1
13/5
14/4
Max
+10.86
+9.30
+27.68
+12.79
+15.04 −1.05
Min
+1.67
+0.17
−0.82
−1.49
ResNet-50
18/0
18/0
18/0
13/5
18/0
Max
+15.19
+10.93
+22.45
+6.98
+13.74
Min
+2.47
+1.07
+0.06
−0.60
+0.15
18/0
17/1
17/1
15/3
16/2
Max
+11.92
+11.01
+14.81
+11.43
+10.94
Min
+1.35
−0.23
−0.02
−0.84
−0.55
18/0
18/0
18/0
16/2
18/0
Max
+19.84
+25.30
+44.16
+20.26
+27.32 +0.18
Wide-ResNet50-2
4.3. PLUS Configuration
ConvNeXt-B
For each class, PLUS selects the m = 16 most class-active neurons from the SAE, orthonormalizes their decoder directions into a basis Qc using sign-fixed QR decomposition, and computes the per-class whitened squared Mahalanobis distance z (with a whitening ridge ϵ = 10−3 ). The supports and the subspace statistics (µc , Σc ) are estimated on a random subset of ID-train images (at most 300 per class). The score z is then standardized and fused with any calibrated base score using a fixed weight β = 0.5.
Min
+1.91
+1.10
+0.24
−0.16
MaxViT-T
18/0
18/0
18/0
13/5
16/2
Max
+17.28
+22.73
+37.43
+20.35
+25.29 −0.40
Min
+1.25
+0.51
+0.11
−0.75
Swin-T
18/0
18/0
16/2
9/9
16/2
Max
+22.41
+26.22
+37.82
+22.97
+29.22 −0.42
Min
+2.53
+0.80
−0.19
−1.62
ViT-B/16
18/0
18/0
16/2
9/9
15/3
Max
+18.48
+21.70
+31.64
+14.77
+23.47
Min
+1.90
+0.38
−0.56
−1.75
−0.78
Table 2. Per-dataset breadth across seven backbones. For each backbone and OOD dataset, the top line counts how many of the 18 baselines improve / drop in AUROC when PLUS is added (improved/dropped; bold marks a clean sweep where all 18 improve). The Max / Min lines give the largest / smallest per-detector AUROC change, in points, over the 18 detectors.
5. Results 5.1. Results on the Three Main Backbones Table 1 evaluates PLUS across all 18 post-hoc baselines on the three standard OpenOOD backbones, comparing the standalone base detectors against their fusion with z at the fixed weight β = 0.5. Integrating z improves the near-OOD AUROC in every one of the 54 detector-backbone combinations (18 detectors × 3 backbones), often by large margins (e.g., +20.1 on GradNorm and +10.9 on Energy with ViT-B/16, and +24.3 on ASH with Swin-T), with FPR95 decreasing correspondingly. Far-OOD and overall AUROC improve in the large majority of cases. The only regressions (underlined) are confined to a few far-OOD metrics on the already far-saturated Mahalanobis-family bases (MDS, RMDS, MDS++, and RMDS++ on ViT-B/16, and MDS++ and RMDS++ on Swin-T) together with ASH on ResNet50; each AUROC drop is at most ≈ 1.0 point.
former backbones, z itself is a weak Textures detector (standalone far-AUROC ≈ 74–78 on Textures versus ≈ 89 on iNaturalist). Because DTD’s pure-texture crops lack object or scene structure, they deviate little within the predicted class’s subspace. Fusing this weak-far channel at a fixed β=0.5 therefore dilutes, rather than sharpens, the alreadystrong base separation on Textures. Nevertheless, overall far-OOD improves in 321 of the 378 cells, and where drops occur they are marginal, never exceeding ≈ 1.8 points. Full per-backbone, per-dataset values are provided in Appendix C.
5.2. Per-Dataset Results Across Seven Backbones
5.3. Comparison to State-of-the-Art
Table 2 breaks down the improvement per dataset and extends it to all seven backbones, counting how many of the 18 detectors improve versus drop on each OOD dataset once PLUS is added. Near-OOD gains are highly uniform: on the hardest split, SSB-hard, PLUS improves all 18 detectors on every backbone. On NINCO, it improves all 18 on six of the seven backbones (the sole exception being a single detector on Wide-ResNet50-2 with a marginal −0.23-point dip). Far-OOD improves broadly but exposes a per-dataset near/far tension. While iNaturalist and OpenImage-O improve for almost every detector, Textures concentrates the trade-off (9/9 improve/drop on ViT-B/16 and Swin-T). This is precisely where z contributes the least: on these trans-
We evaluate an exemplary configuration, PLUS instantiated as MDS++ fused with 0.5z, where our base-agnostic complement is applied to a strong base detector. We compare it against the two strongest baselines per split (near- and farAUROC), as summarized in Table 3. Near-OOD
Backbone MDS++ + PLUS ViT-B/16 ResNet-50 Swin-T
81.09 78.23 80.86
Best baselines RMDS++ 80.07, RMDS 79.99 ASH 78.17, NNGuide 77.53 RMDS++ 79.70, GEN 78.97
Far-OOD MDS++ + PLUS 92.26 95.45 93.00
Best baselines MDS++ 93.29, ViM 92.93 ASH 95.74, MDS++ 95.13 MDS++ 93.67, ViM 93.26
Table 3. Comparison with state-of-the-art methods in AUROC. In each Near- and Far-OOD block, the best result is in bold, and the second-best is underlined.
6
MDS++ + PLUS achieves a higher near-OOD AUROC than every standalone baseline on all three backbones, exceeding even the strongest Mahalanobis variant (RMDS++) on ViT-B/16 and Swin-T, and edging the strongest baseline on ResNet-50 (ASH; 78.23 vs. 78.17). On far-OOD, performance is slightly below the strongest baseline (MDS++ on ViT-B/16 and Swin-T), reflecting the small, bounded farOOD cost of incorporating the near-OOD signal z. Other bases fused with PLUS reach even higher near-OOD AUROC (e.g., ASH + PLUS 81.34 on ResNet-50, Table 1).
thus the more consistent choice across backbones: on all seven backbones, its largest shortfall from the best subspace is 0.93 points, versus 1.91 for per-class PCA (Appendix D).
6.2. Sensitivity to the Fusion Weight β Figure 2 sweeps the fusion weight β over an MDS++ base. Adding the z complement improves near-OOD universally, but the gain profile is backbone-dependent: on ViT-B/16 and Swin-T, near-AUROC saturates around β ≈ 0.5, whereas on ResNet-50 and ConvNeXt-B it continues to rise. In contrast, far-OOD performance typically peaks near β=0.25 before declining. Thus, on most backbones, weights beyond 0.5 yield little additional near-OOD benefit while steadily eroding far-OOD accuracy.
6. Analysis 6.1. Where Does the Near-OOD Gain Come From? z combines a per-class subspace (Sc ) and a per-class metric (Σc ). We ablate both independently (Table 4). Subspace
Metric
ViT-B/16
ResNet-50
Swin-T
SAE top-16 (ours)
per-class Σc tied isotropic
81.23 80.61 80.33
78.22 74.79 75.46
80.90 80.35 80.04
Random orthonormal
per-class Σc tied isotropic
79.57 79.01 79.00
74.47 74.20 74.20
78.49 77.88 77.88
Full-space per-class Ledoit–Wolf [17] MDS++ alone (no z)
80.52 79.87
75.58 74.71
79.11 78.92
Table 4. Subspace × metric ablation (near-OOD AUROC of MDS++ + 0.5 z, %). Rows vary the 16-dim per-class subspace and the metric applied inside it. Tied: a single within-class covariance shared by all classes, projected into each class’s subspace; isotropic: unwhitened ∥D∥2 . Random orthonormal: mean over 5 seeds (std ≤ 0.13). Ledoit–Wolf replaces z by a per-class shrinkage Mahalanobis on the full feature.
Figure 2. Sensitivity to the fusion weight β on an MDS++ base. Near-AUROC (solid) and far-AUROC (dashed) across backbones.
We adopt a fixed β=0.5 as a conservative, base-favored default (a 2:1 trust ratio). While β=0.25 might offer a marginally better far-OOD trade-off depending on the backbone, we exclude OOD-based tuning. This fixed value robustly secures near-gains across all seven backbones (Section 5.2) while avoiding the far-OOD penalties of larger weights.
Neither ingredient suffices alone. With a random 16dimensional subspace, the per-class metric yields no nearOOD gain over MDS++ alone. Conversely, within the SAE subspace, a shared (tied) or isotropic metric recovers only part of the gain, and the tied metric gives almost no benefit on ResNet-50. Only their combination yields the full improvement (+1.36 to +3.51 points over MDS++). The near-OOD signal thus requires modeling perclass anisotropy along the directions in which that class is expressed.
6.3. Sensitivity to the Support Size m The support size is fixed at m=16 for all backbones (Section 4). This sweep examines whether a smaller support would have been preferable: across all backbones, the fused near-OOD gain increases up to m=16, and reducing the support offers no Pareto improvement. On ViT-B/16 and Swin-T, setting m∈[2, 4] recovers a fraction of far-OOD performance only at the steep expense of near-OOD gains. On ResNet-50, performance outright collapses: 11/18 detectors suffer far-OOD drops at m=2, compared to a clean sweep of improvements at m=16. This sensitivity to support size is fundamentally tied to a backbone’s feature participation ratio PR = P 2 P 2 i λi i λi , computed over the eigenvalues λi of the ID-train feature covariance (PR is high when variance
Why an SAE supplies the subspace. Under the same perclass metric, 16 randomly chosen SAE neurons yield essentially no gain over MDS++ (79.34, 74.83, and 78.00 near-AUROC), so the benefit comes from selecting each class’s most active neurons. Per-class PCA, the natural SAE-free alternative, is slightly ahead on ViT-B/16 and Swin-T (81.64 and 81.42) but drops on ResNet-50 (76.69 vs. 78.22), likely because it estimates each class’s directions from that class’s samples alone. The SAE support is 7
AUROC in every one of the 714 base–partner pairs across seven backbones (+1.7 to +5.1 points on average). The information in z is therefore not supplied by combining existing detectors, which is consistent with z being only partially correlated with the base scores (Spearman ρ ≤ 0.83).
7. Conclusion We present PLUS, a detector-agnostic complement for nearOOD detection that computes a per-class subspace deviation z from a fixed sparse autoencoder support. Fused additively with any calibrated base score under a shared protocol, the signal z enables robust near-OOD discrimination. Across seven distinct backbones and 18 post-hoc detectors spanning parametric, non-parametric, logit-based, prototype, and hybrid families, adding z consistently improves near-OOD AUROC for every evaluated combination while broadly preserving far-OOD performance. Crucially, these gains require no network retraining, architectural modifications, or exposure to auxiliary OOD data. This empirical improvement is consistent with our geometric characterization: while near-OOD inputs remain ID-like under global feature summaries, they exhibit measurable deviation within their predicted class’s low-dimensional subspace. The subspace signal z captures this withinmanifold cue precisely where global scores lose discriminative power, thereby complementing far-saturated detectors at a bounded far-OOD cost. PLUS integrates this geometric insight with sparse-autoencoder-based per-class subspaces, scoring them via a unified, detector-agnostic protocol. This yields a simple, reproducible, and effective framework for reliable OOD deployment.
Figure 3. Support-size demand tracks the feature participation ratio. The small-m Near-AUROC loss (fused near-gain at m=16 minus m=2, in points) against the ID-train participation ratio PR, over seven backbones.
is spread over many directions, low when a few dominate). As Figure 3 shows, the near-OOD penalty for using a small m falls systematically as PR rises: the two lowest-PR backbones incur by far the largest penalty and require the full m=16, whereas the high-PR transformers and ConvNeXt-B lose little at small m and accommodate m=16 without compromising near-OOD gains. Mean near gain A+z
A+B
ViT-B/16 +6.48 ResNet-50 +4.83 Swin-T +6.55 ResNet-18 +3.79 Wide-ResNet50-2 +3.12 ConvNeXt-B +7.75 MaxViT-T +5.84
+3.88 +1.86 +4.14 +1.66 +2.92 +4.90 +4.22
Backbone
PLUS >
z on top of A+B near / far ∆
MDS++ partner pairs improved 15/17 17/17 14/17 16/17 11/17 16/17 9/17
102/102 102/102 102/102 102/102 102/102 102/102 102/102
+4.22 / +2.15 +3.43 / +1.75 +4.09 / +2.41 +2.81 / +1.92 +1.71 / +0.90 +5.09 / +6.25 +3.58 / +2.67
Future Work. Natural next steps include generalizing per-class subspace construction beyond ImageNet to smallscale datasets as well as enterprise-scale settings with massive label spaces. Furthermore, extending this framework from image-level classification to dense prediction tasks, such as object detection, offers a promising direction. From a deployment perspective, replacing the fixed weighting with a principled, sample-adaptive fusion mechanism without auxiliary OOD exposure would further maximize practical sensitivity. Finally, establishing a rigorous mathematical framework for subspace separability will clarify the fundamental conditions under which per-class deviations yield near-OOD discriminability.
Table 5. Ensemble control (near-OOD AUROC). Mean near gain: over the 18 bases A, the gain of A+z (PLUS) and of A+B averaged over the available partners B ̸= A, both relative to A alone. PLUS > MDS++ partner: number of bases (A ̸= MDS++) for which A+z beats A+MDS++. z on top of A+B: adding 0.5 cz to every two-detector ensemble cA + 0.5 cB .
6.4. Complementarity Beyond Ensembling Fusing two detectors often helps on its own, so we test whether z adds something unique by replacing it in the fusion with an existing detector B (cA + 0.5 cB , same calibration) for six strong partners B ∈ {MDS++, RMDS++, ViM, KNN, GEN, ASH} (Table 5). On every backbone, adding z improves a base more than adding an existing detector does on average, and it beats a strong partner, MDS++, for a majority of bases (9–17 out of 17). More importantly, adding z on top of a two-detector ensemble improves near-OOD
Acknowledgments This work was supported by the National Research Foundation of Korea grant (No. RS-2026-25490405) and by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant (No. RS-2025-02214652) funded by the Korea government (MSIT).
8
References
networks. In International Conference on Learning Representations (ICLR), 2017. 2, 5 [14] Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joseph Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. In International Conference on Machine Learning (ICML), 2022. 2, 5 [15] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. In Advances in Neural Information Processing Systems (NeurIPS), 2021. 3, 5 [16] Donghwan Kim and Hyunsoo Yoon. MahaVar: OOD detection via class-wise Mahalanobis distance variance under neural collapse. In Advances in Neural Information Processing Systems (NeurIPS), 2026. 3 [17] Olivier Ledoit and Michael Wolf. A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis, 88(2), 2004. 7 [18] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems (NeurIPS), 2018. 3, 5 [19] Litian Liu and Yao Qin. Fast decision boundary based outof-distribution detector. In International Conference on Machine Learning (ICML), 2024. 3, 5 [20] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. In Advances in Neural Information Processing Systems (NeurIPS), 2020. 2, 5 [21] Xixi Liu, Yaroslava Lochman, and Christopher Zach. GEN: Pushing the limits of softmax-based out-of-distribution detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2, 5 [22] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 4 [23] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A ConvNet for the 2020s. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 5 [24] Aleksandar Makelov, Georg Lange, and Neel Nanda. Towards principled evaluations of sparse autoencoders for interpretability and control. In International Conference on Learning Representations (ICLR), 2025. 3 [25] Maximilian Mueller and Matthias Hein. Mahalanobis++: Improving OOD detection via feature normalization. In International Conference on Machine Learning (ICML), 2025. 3, 5 [26] Ahyoung Oh, Wonseok Shin, and Songkuk Kim. Sparsity as a key: Unlocking new insights from latent structures for out-of-distribution detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. 3 [27] Jaewoo Park, Yoon Gyo Jung, and Andrew Beng Jin Teoh. Nearest neighbor guidance for out-of-distribution detection. In IEEE/CVF International Conference on Computer Vision (ICCV), 2023. 3, 5
[1] Mouı̈n Ben Ammar, Nacim Belkhir, Sebastian Popescu, Antoine Manzanera, and Gianni Franchi. NECO: Neural collapse based out-of-distribution detection. In International Conference on Learning Representations (ICLR), 2024. 3 [2] Julian Bitterwolf, Maximilian Müller, and Matthias Hein. In or out? Fixing ImageNet out-of-distribution detection evaluation. In International Conference on Machine Learning (ICML), 2023. 4 [3] Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. https : / / transformer - circuits . pub / 2023 / monosemantic-features. 3 [4] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2014. 4 [5] Matthew Cook, Alina Zare, and Paul Gader. Outlier detection through null space analysis of neural networks. In ICML Workshop on Uncertainty and Robustness in Deep Learning, 2020. 3 [6] Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In International Conference on Learning Representations (ICLR), 2024. 3 [7] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2009. 4 [8] Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for outof-distribution detection. In International Conference on Learning Representations (ICLR), 2023. 3, 5 [9] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021. 4 [10] Kun Fang, Qinghua Tao, Kexin Lv, Mingzhen He, Xiaolin Huang, and Jie Yang. Kernel PCA for out-of-distribution detection. In Advances in Neural Information Processing Systems (NeurIPS), 2024. 3 [11] Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. In International Conference on Learning Representations (ICLR), 2025. 3, 5 [12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 4 [13] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural
9
[41] Jinsong Zhang, Qiang Fu, Xu Chen, Lun Du, Zelin Li, Gang Wang, Xiaoguang Liu, Shi Han, and Dongmei Zhang. Outof-distribution detection based on in-distribution data patterns memorization with modern Hopfield energy. In International Conference on Learning Representations (ICLR), 2023. 3, 5 [42] Jingyang Zhang, Jingkang Yang, Pengyun Wang, Haoqi Wang, Yueqian Lin, Haoran Zhang, Yiyou Sun, Xuefeng Du, Yixuan Li, Ziwei Liu, Yiran Chen, and Hai Li. OpenOOD v1.5: Enhanced benchmark for out-of-distribution detection. Journal of Data-centric Machine Learning Research (DMLR), 2024. 4
[28] Gonçalo Paulo and Nora Belrose. Sparse autoencoders trained on the same data learn different features. In International Conference on Learning Representations (ICLR), 2026. 3 [29] Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. Improving sparse decomposition of language model activations with gated sparse autoencoders. In Advances in Neural Information Processing Systems (NeurIPS), 2024. 3 [30] Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with JumpReLU sparse autoencoders. arXiv preprint arXiv:2407.14435, 2024. 3 [31] Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan. A simple fix to Mahalanobis distance for improving near-OOD detection. arXiv preprint arXiv:2106.09022, 2021. 3, 5 [32] Yiyou Sun and Yixuan Li. DICE: Leveraging sparsification for out-of-distribution detection. In European Conference on Computer Vision (ECCV), 2022. 3, 5 [33] Yiyou Sun, Chuan Guo, and Yixuan Li. ReAct: Out-ofdistribution detection with rectified activations. In Advances in Neural Information Processing Systems (NeurIPS), 2021. 3, 5 [34] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-ofdistribution detection with deep nearest neighbors. In International Conference on Machine Learning (ICML), 2022. 3, 5 [35] Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, et al. Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet. Transformer Circuits Thread, 2024. https: //transformer-circuits.pub/2024/scalingmonosemanticity/. 3 [36] Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. MaxViT: Multi-axis vision transformer. In European Conference on Computer Vision (ECCV), 2022. 5 [37] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The iNaturalist species classification and detection dataset. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 4 [38] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recognition: A good closed-set classifier is all you need? In International Conference on Learning Representations (ICLR), 2022. 4 [39] Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. ViM: Out-of-distribution with virtual-logit matching. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 3, 4, 5 [40] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference (BMVC), 2016. 4
10
Supplementary Material Don’t Replace, Complement: A Detector-Agnostic Subspace Signal for Near-OOD Detection
A. Backbone Checkpoints d
Backbone
Family
torchvision checkpoint
ResNet-18 ResNet-50 Wide-ResNet50-2 ConvNeXt-B Swin-T ViT-B/16 MaxViT-T
CNN, BasicBlock CNN, Bottleneck CNN, Bottleneck (2× width) ConvNeXt Swin Transformer ViT Hybrid Conv-Transformer
ResNet18 Weights.IMAGENET1K V1 ResNet50 Weights.IMAGENET1K V1 Wide ResNet50 2 Weights.IMAGENET1K V1 ConvNeXt Base Weights.IMAGENET1K V1 Swin T Weights.IMAGENET1K V1 ViT B 16 Weights.IMAGENET1K V1 MaxVit T Weights.IMAGENET1K V1
512 2048 2048 1024 768 768 512
OpenOOD ✓
✓ ✓
Table 6. Backbone checkpoints. All weights are torchvision IMAGENET1K V1 public checkpoints, used without fine-tuning; d is the penultimate-feature (pre-logit) dimension. The three ResNet-family backbones differ structurally: ResNet-18 uses shallow BasicBlock residual units (d{=}512), whereas ResNet-50 and Wide-ResNet50-2 use the deeper Bottleneck design (d{=}2048), the latter doubling the bottleneck width. Backbones marked ✓ are the three standard OpenOOD v1.5 ImageNet backbones.
B. Neuron Count p: Sensitivity and Divergence To evaluate the effect of fixing p = 6144, we sweep the SAE neuron count from p = 2d to p = 12d across seven backbones, extending it beyond 12d for ResNet-18 (up to 64d, as its response keeps increasing), MaxViT-T, and Swin-T (up to 24d). Holding other hyperparameters constant, we measure the z-standalone near- and far-OOD AUROC using ID-validation negatives against each OOD set. Table 7 and Figure 4 detail each backbone’s optimal capacity and the performance gaps compared to p = 6144. Minimal standalone cost from fixing p = 6144. The near-OOD performance is generally stable with respect to p. Four of the seven backbones stay within ±0.5 AUROC points of their performance at the deployed p across the entire sweep. Although ResNet-18 and MaxViT-T show directional trends, the largest performance gap from fixing p = 6144 within a practical overcomplete range (≤ 12×) is +0.98 for MaxViT-T. While larger capacities (> 12×) can yield marginal improvements for certain backbones, such sizes are computationally inefficient. Performance gaps further narrow under fusion. When evaluating the fused scores (base detector + 0.5z) across all 126 combinations, tuning p per backbone improves the near-OOD AUROC by only +0.23 on average (Table 7). Standalone gains are substantially reduced after fusion (e.g., the +0.98 gap for MaxViT-T decreases to +0.19). Furthermore, tuning p specifically for near-OOD can degrade far-OOD performance, as observed in ViT-B/16. Consequently, backbone-specific tuning provides negligible fused gains at the cost of making a deployment constant dependent on OOD access. Backbone ResNet-18 ResNet-50 Wide-ResNet50-2 ConvNeXt-B MaxViT-T Swin-T ViT-B/16
d
Near-opt p (×)
∆near standalone
∆near fused
Far-opt p (×)
512 2048 2048 1024 512 768 768
24576 (48×) 6144 (3×) 8192 (4×) 4096 (4×) 1536 (3×) 9216 (12×) 3072 (4×)
+1.35 +0.00 +0.47 +0.29 +0.98 +0.41 +0.35
+0.75 +0.00 +0.20 +0.12 +0.19 +0.20 +0.15
2048 (4×) 4096 (2×) 4096 (2×) 4096 (4×) 3072 (6×) 15360 (20×) 6144 (8×)
Table 7. Per-backbone optimal SAE capacity. Near-opt and Far-opt p indicate the capacities maximizing standalone near- and far-OOD AUROC, respectively. ∆near std and fused represent the near-AUROC gained by using the near-optimal p instead of the fixed p = 6144, evaluated as z standalone (standalone) and after base+0.5z fusion (fused).
11
Figure 4. Near-AUROC response to the neuron count p (∆ relative to the deployed p = 6144; shared vertical scale, grey band = ±0.5 pt). Four backbones remain inside the noise band (flat). ResNet-50 dips once at 8× but peaks at the deployed p, while ResNet-18 (which seeks capacity) and MaxViT-T (which prefers compactness) diverge from the band in opposite directions. Top axis indicates the expansion factor p/d.
Divergent optima for near- and far-OOD. For six of the seven backbones, the optimal p for near-OOD differs from that for far-OOD (Table 7, Fig. 5). This divergence follows architectural lines: standard CNNs favor a smaller p for far-OOD detection, whereas transformers and hybrids benefit from a larger p. Increasing p allows the per-class support to capture finer ID-specific directions, which improves near-OOD separation but can overfit and degrade far-OOD performance (e.g., ResNet-18 is near-optimal at 48× but far-optimal at 4×). ConvNeXt-B is the sole exception where these optima align. Ultimately, the fixed capacity of p = 6144 serves as an effective compromise between these competing optima.
Figure 5. Near (blue) and far (orange) z-AUROC vs. neuron count. Stars mark each metric’s optimum, while the red dashed line indicates the deployed p = 6144. The near- and far-optima diverge for six of the seven backbones; CNNs prefer a smaller p for far-OOD, whereas transformers prefer a larger one.
Summary. We fix p = 6144 across all backbones. The standalone performance concession is at most 0.98 near-AUROC points within 12× (1.35 for ResNet-18 at 48×), and this gap shrinks to +0.23 on average after fusion. 12
C. Full Experimental Results For each backbone, all 18 baselines are reported both standalone and with PLUS (shaded, bold), under the same protocol as Table 1. An underline marks a per-metric performance regression. Method
SSB-hard
NINCO
Near-OOD Average
iNaturalist
Textures
OpenImage-O
Far-OOD Average
AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ MSP MSP + PLUS MaxLogit MaxLogit + PLUS Energy Energy + PLUS GEN GEN + PLUS KL-Matching KL-Matching + PLUS ReAct ReAct + PLUS ASH ASH + PLUS DICE DICE + PLUS ViM ViM + PLUS KNN KNN + PLUS MDS MDS + PLUS RMDS RMDS + PLUS MDS++ MDS++ + PLUS RMDS++ RMDS++ + PLUS GradNorm GradNorm + PLUS SHE SHE + PLUS NNGuide NNGuide + PLUS fDBD fDBD + PLUS
68.94 73.66 64.20 71.73 58.80 69.80 70.09 74.07 68.14 71.20 63.10 70.62 53.90 68.66 59.10 68.97 69.39 73.18 65.80 71.62 71.56 74.24 72.77 74.94 72.56 74.59 73.06 74.95 42.96 61.43 67.96 72.48 53.39 65.65 65.79 71.54
86.42 79.37 91.52 85.92 92.24 88.72 82.23 79.37 88.35 84.39 90.46 84.89 93.50 89.78 89.72 83.67 90.18 86.63 86.48 84.61 83.49 81.55 84.89 82.04 84.07 82.46 84.41 82.30 93.62 91.33 85.69 84.12 93.20 89.38 86.91 83.70
78.11 82.60 72.40 79.57 66.02 76.75 82.51 85.28 80.68 82.88 75.43 81.67 52.51 72.35 71.73 80.30 84.66 86.69 82.06 85.55 86.50 87.64 87.20 88.00 87.19 87.58 87.08 87.46 35.60 57.31 84.07 86.39 61.19 74.67 80.76 84.48
77.27 58.22 92.94 82.84 94.13 90.24 59.26 52.31 66.09 56.64 78.47 64.44 95.33 87.96 80.88 62.12 57.08 49.32 54.82 49.65 48.72 46.01 46.54 44.28 47.17 45.19 47.25 44.77 95.81 93.52 56.47 48.81 90.60 81.84 60.36 54.84
73.53 78.13 68.30 75.65 62.41 73.28 76.30 79.67 74.41 77.04 69.27 76.15 53.20 70.50 65.42 74.63 77.03 79.94 73.93 78.59 79.03 80.94 79.98 81.47 79.88 81.09 80.07 81.20 39.28 59.37 76.01 79.44 57.29 70.16 73.28 78.01
81.84 68.80 92.23 84.38 93.19 89.48 70.75 65.84 77.22 70.52 84.47 74.66 94.41 88.87 85.30 72.89 73.63 67.97 70.65 67.13 66.10 63.78 65.72 63.16 65.62 63.82 65.83 63.53 94.72 92.42 71.08 66.47 91.90 85.61 73.63 69.27
88.19 91.70 85.29 90.57 79.30 89.01 93.54 95.04 89.59 91.30 86.11 91.93 50.62 78.96 82.79 91.70 95.74 96.36 91.27 94.12 96.03 96.32 96.09 96.23 96.19 95.63 95.78 95.27 42.42 74.06 93.53 94.73 74.51 88.83 90.51 93.58
42.41 35.01 72.94 43.74 83.56 56.09 22.92 21.27 43.48 34.78 48.25 33.63 97.02 86.17 47.16 29.13 17.63 15.58 28.37 23.21 20.57 17.08 19.61 17.42 16.81 18.23 18.35 19.77 91.16 87.56 22.49 20.52 74.06 48.23 32.76 25.88
85.06 85.74 83.74 85.40 81.17 84.07 90.23 89.81 86.49 86.97 86.66 87.26 48.53 62.66 82.46 84.84 90.90 90.42 91.14 89.71 89.53 88.85 89.42 88.60 90.94 89.20 89.88 88.27 44.99 59.76 92.62 91.51 77.10 82.20 88.52 88.06
56.46 52.74 78.94 64.51 83.66 74.92 38.30 38.82 50.13 46.37 55.88 50.48 98.50 96.62 54.65 46.38 40.34 36.58 33.34 33.60 38.95 39.60 37.32 38.42 31.74 34.82 35.09 37.23 92.25 91.94 26.30 29.70 72.34 63.56 44.86 43.67
84.86 87.88 81.60 86.35 76.48 84.50 90.27 91.53 87.03 88.49 84.29 88.53 55.51 74.89 82.47 88.26 92.16 92.68 89.73 91.32 92.40 92.47 92.32 92.26 92.73 91.96 92.25 91.52 37.82 61.29 90.98 91.93 71.26 82.83 89.06 90.86
Table 8. ViT-B/16 — per-dataset AUROC / FPR95, standalone vs. PLUS.
13
56.16 43.83 85.82 61.96 88.82 76.85 35.47 31.49 51.69 40.57 57.67 44.62 94.79 86.91 52.13 39.10 29.50 26.73 35.13 28.29 30.29 27.59 29.45 27.74 26.24 26.69 27.58 27.91 94.53 93.35 33.81 28.29 76.98 61.25 39.36 31.86
86.04 88.44 83.54 87.44 78.98 85.86 91.35 92.13 87.70 88.92 85.69 89.24 51.55 72.17 82.57 88.27 92.93 93.15 90.71 91.72 92.65 92.55 92.61 92.36 93.29 92.26 92.64 91.69 41.74 65.04 92.38 92.72 74.29 84.62 89.36 90.83
51.68 43.86 79.23 56.74 85.35 69.29 32.23 30.53 48.43 40.57 53.93 42.91 96.77 89.90 51.31 38.20 29.16 26.30 32.28 28.37 29.94 28.09 28.79 27.86 24.93 26.58 27.01 28.30 92.65 90.95 27.53 26.17 74.46 57.68 38.99 33.80
Method
SSB-hard
NINCO
Near-OOD Average
iNaturalist
Textures
OpenImage-O
Far-OOD Average
AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ MSP MSP + PLUS MaxLogit MaxLogit + PLUS Energy Energy + PLUS GEN GEN + PLUS KL-Matching KL-Matching + PLUS ReAct ReAct + PLUS ASH ASH + PLUS DICE DICE + PLUS ViM ViM + PLUS KNN KNN + PLUS MDS MDS + PLUS RMDS RMDS + PLUS MDS++ MDS++ + PLUS RMDS++ RMDS++ + PLUS GradNorm GradNorm + PLUS SHE SHE + PLUS NNGuide NNGuide + PLUS fDBD fDBD + PLUS
72.09 75.98 72.51 77.97 72.08 77.98 72.01 77.67 71.38 73.85 73.02 77.76 72.89 77.77 70.14 77.54 65.47 72.79 59.47 68.86 48.51 63.70 71.45 76.47 66.42 72.41 70.12 75.55 71.90 78.54 70.96 77.94 73.01 78.04 70.66 75.42
74.48 71.61 76.19 71.35 76.54 71.69 75.72 71.81 84.76 77.89 77.57 73.38 73.66 70.64 77.93 73.12 80.57 76.42 86.16 81.33 92.12 85.79 78.26 74.53 82.97 78.83 80.52 76.16 78.23 70.46 76.48 70.41 75.24 71.34 77.29 74.01
79.95 82.61 80.41 83.75 79.70 83.43 81.70 84.54 81.89 83.18 81.73 83.98 83.45 84.91 76.03 81.21 78.61 81.95 77.66 81.75 62.38 73.31 81.97 84.52 82.99 84.05 81.37 84.25 74.02 79.27 76.45 81.35 82.05 84.16 82.61 84.60
56.76 55.04 59.41 55.00 60.57 55.45 54.85 52.46 60.27 53.70 55.84 53.11 52.96 51.98 66.87 62.00 61.90 58.57 62.32 59.66 78.56 69.64 52.40 49.80 55.13 54.16 58.27 54.18 79.47 72.11 69.76 63.03 56.57 53.79 52.08 51.32
76.02 79.30 76.46 80.86 75.89 80.71 76.86 81.11 76.63 78.52 77.38 80.87 78.17 81.34 73.09 79.38 72.04 77.37 68.56 75.31 55.45 68.50 76.71 80.50 74.70 78.23 75.75 79.90 72.96 78.91 73.70 79.64 77.53 81.10 76.63 80.01
65.62 63.33 67.80 63.17 68.56 63.57 65.28 62.14 72.52 65.80 66.70 63.24 63.31 61.31 72.40 67.56 71.23 67.50 74.24 70.50 85.34 77.72 65.33 62.16 69.05 66.50 69.39 65.17 78.85 71.28 73.12 66.72 65.91 62.56 64.69 62.67
88.41 92.69 91.16 95.24 90.63 95.23 92.44 95.56 90.78 92.35 96.34 96.59 97.06 97.13 92.55 95.56 89.44 94.30 86.70 93.70 63.33 85.78 86.42 93.88 94.91 95.73 89.79 94.60 93.89 96.04 92.56 95.66 95.84 96.50 93.71 95.86
43.34 33.58 30.62 21.10 31.34 21.10 26.11 19.99 38.54 31.96 16.73 15.13 14.10 13.02 33.42 21.09 30.93 24.08 42.11 27.23 74.12 50.24 36.30 25.68 20.04 19.02 36.89 24.07 32.04 20.70 34.24 21.66 18.76 15.87 22.01 18.86
82.43 87.62 88.39 91.74 88.70 91.77 87.60 91.20 84.70 87.00 92.79 93.40 96.90 96.30 92.04 93.45 97.95 97.59 97.53 97.51 89.80 93.70 86.65 91.01 98.34 97.88 81.18 88.17 92.05 94.08 93.50 95.12 95.22 95.03 93.45 94.51
60.88 46.06 46.11 34.08 45.76 33.62 46.24 33.90 52.94 41.34 29.63 26.09 15.28 14.68 44.10 31.03 10.59 10.93 14.19 11.34 42.62 28.56 47.45 36.18 8.78 9.60 69.42 45.98 43.28 30.42 36.22 23.79 25.88 20.68 27.69 22.93
84.86 88.62 89.17 91.95 89.06 91.86 89.26 91.97 87.30 88.84 91.87 92.26 93.26 93.40 88.26 90.97 90.45 92.51 86.67 91.21 69.16 82.90 85.45 90.11 92.13 92.75 84.69 89.87 84.83 88.70 86.48 90.23 92.70 92.97 91.18 92.85
50.13 43.01 37.86 32.43 38.08 32.29 34.51 30.62 48.96 40.54 32.56 31.84 29.18 28.23 47.90 39.71 33.06 29.01 46.45 35.04 72.18 54.84 41.55 35.08 32.30 30.03 49.65 37.09 68.42 55.63 55.23 45.02 30.93 29.50 29.92 28.14
85.23 89.64 89.57 92.98 89.46 92.95 89.77 92.91 87.59 89.40 93.67 94.08 95.74 95.61 90.95 93.33 92.61 94.80 90.30 94.14 74.10 87.46 86.17 91.67 95.13 95.45 85.22 90.88 90.26 92.94 90.85 93.67 94.59 94.83 92.78 94.41
51.45 40.88 38.20 29.20 38.39 29.00 35.62 28.17 46.81 37.95 26.31 24.35 19.52 18.64 41.81 30.61 24.86 21.34 34.25 24.54 62.97 44.55 41.77 32.31 20.37 19.55 51.99 35.71 47.91 35.58 41.90 30.16 25.19 22.02 26.54 23.31
Table 9. ResNet-50 — per-dataset AUROC / FPR95, standalone vs. PLUS.
Method
SSB-hard
NINCO
Near-OOD Average
iNaturalist
Textures
OpenImage-O
Far-OOD Average
AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ MSP MSP + PLUS MaxLogit MaxLogit + PLUS Energy Energy + PLUS GEN GEN + PLUS KL-Matching KL-Matching + PLUS ReAct ReAct + PLUS ASH ASH + PLUS DICE DICE + PLUS ViM ViM + PLUS KNN KNN + PLUS MDS MDS + PLUS RMDS RMDS + PLUS MDS++ MDS++ + PLUS RMDS++ RMDS++ + PLUS GradNorm GradNorm + PLUS SHE SHE + PLUS NNGuide NNGuide + PLUS fDBD fDBD + PLUS
71.78 75.49 70.44 75.21 68.21 74.56 72.75 76.18 70.24 72.85 69.30 74.50 45.87 68.28 47.89 63.25 68.82 73.27 63.87 71.22 68.62 73.05 71.64 74.87 72.11 74.96 73.17 75.69 50.36 65.19 70.43 74.67 56.26 67.15 65.93 71.97
80.90 75.00 86.52 81.67 87.45 83.33 79.19 75.97 85.32 81.20 85.02 80.12 95.70 90.25 96.39 89.30 88.62 83.39 85.35 82.68 83.78 80.46 82.98 79.67 83.16 81.00 81.90 79.73 93.57 91.60 85.27 82.19 94.26 90.21 86.23 82.69
81.72 84.55 80.89 84.37 78.25 83.34 85.18 87.00 81.98 83.71 81.97 85.13 47.07 73.30 49.02 65.61 81.80 84.58 78.92 83.83 81.77 84.35 84.81 86.51 85.73 86.76 86.23 87.04 44.81 63.60 82.54 85.79 65.32 76.01 80.12 83.81
61.20 53.65 74.94 62.32 79.03 67.86 48.01 45.58 65.59 54.51 60.28 51.98 94.37 84.46 97.35 90.11 60.62 51.19 58.68 52.85 53.66 50.08 50.39 47.85 52.16 48.98 51.17 48.19 94.02 92.07 67.84 57.28 89.18 79.11 59.24 52.39
76.75 80.02 75.66 79.79 73.23 78.95 78.97 81.59 76.11 78.28 75.63 79.81 46.47 70.79 48.45 64.43 75.31 78.92 71.39 77.53 75.19 78.70 78.22 80.69 78.92 80.86 79.70 81.37 47.59 64.39 76.49 80.23 60.79 71.58 73.03 77.89
71.05 64.33 80.73 72.00 83.24 75.59 63.60 60.77 75.45 67.86 72.65 66.05 95.03 87.35 96.87 89.70 74.62 67.29 72.02 67.77 68.72 65.27 66.69 63.76 67.66 64.99 66.53 63.96 93.79 91.83 76.56 69.73 91.72 84.66 72.73 67.54
89.86 92.96 88.99 92.82 85.13 91.97 94.23 95.72 90.92 92.34 90.03 94.01 46.62 84.44 47.07 74.41 94.65 96.15 88.63 93.90 93.60 95.46 95.50 96.37 96.25 96.23 96.27 96.08 38.23 71.54 92.49 94.49 76.64 88.99 90.91 94.26
37.31 31.73 49.20 31.40 61.52 35.75 20.51 18.30 34.96 30.30 31.55 24.31 94.24 70.80 98.17 86.93 17.61 15.84 31.71 24.30 21.82 18.81 18.52 16.78 15.92 16.28 16.34 16.95 95.12 90.39 34.84 23.90 72.93 48.78 27.36 23.25
83.27 84.31 81.68 83.31 78.96 81.43 88.15 87.89 86.04 86.70 86.99 86.74 41.66 64.63 77.35 82.84 92.77 92.36 90.59 89.64 89.85 89.82 89.21 89.03 91.69 90.06 88.56 87.28 34.71 49.18 88.83 89.09 89.27 89.89 88.35 87.96
60.88 53.85 79.82 68.55 84.45 75.94 44.58 42.32 49.12 46.30 54.67 47.64 96.16 91.40 87.70 73.12 29.46 31.33 36.16 34.66 37.43 37.82 39.17 38.92 33.33 33.48 39.48 39.42 97.73 97.23 51.55 44.18 50.34 43.22 42.30 40.97
85.77 88.36 83.72 87.24 79.86 85.39 90.56 91.74 87.99 89.26 87.66 89.90 45.64 74.86 57.62 73.70 92.34 93.33 88.51 91.32 91.07 92.39 92.11 92.84 93.08 92.70 92.48 92.06 33.47 55.22 87.53 90.02 77.77 85.23 89.48 91.39
Table 10. Swin-T — per-dataset AUROC / FPR95, standalone vs. PLUS.
14
49.30 43.31 74.43 54.52 80.79 64.92 32.69 30.63 45.90 38.95 42.81 35.94 93.72 82.75 96.53 86.96 26.50 24.92 36.15 29.16 30.65 27.85 29.13 27.49 26.27 25.96 27.20 27.34 96.99 95.93 54.92 40.44 72.85 57.81 35.00 30.02
86.30 88.54 84.80 87.79 81.32 86.26 90.98 91.78 88.32 89.43 88.23 90.22 44.64 74.64 60.68 76.98 93.25 93.95 89.24 91.62 91.51 92.56 92.27 92.75 93.67 93.00 92.44 91.81 35.47 58.65 89.62 91.20 81.23 88.04 89.58 91.20
49.16 42.96 67.82 51.49 75.59 58.87 32.59 30.42 43.33 38.52 43.01 35.96 94.71 81.65 94.13 82.34 24.52 24.03 34.67 29.37 29.97 28.16 28.94 27.73 25.17 25.24 27.67 27.90 96.61 94.52 47.10 36.17 65.37 49.94 34.89 31.41
SSB-hard
Method
NINCO
Near-OOD Average
iNaturalist
Textures
OpenImage-O
Far-OOD Average
AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ MSP MSP + PLUS MaxLogit MaxLogit + PLUS Energy Energy + PLUS GEN GEN + PLUS KL-Matching KL-Matching + PLUS ReAct ReAct + PLUS ASH ASH + PLUS DICE DICE + PLUS ViM ViM + PLUS KNN KNN + PLUS MDS MDS + PLUS RMDS RMDS + PLUS MDS++ MDS++ + PLUS RMDS++ RMDS++ + PLUS GradNorm GradNorm + PLUS SHE SHE + PLUS NNGuide NNGuide + PLUS fDBD fDBD + PLUS
67.37 70.67 65.46 70.65 64.82 70.51 64.82 70.41 67.39 69.06 64.66 69.76 65.47 70.32 60.74 67.99 60.00 66.52 57.55 64.06 47.96 58.82 67.14 71.53 63.66 67.85 65.98 70.52 66.30 71.05 63.21 69.05 64.98 70.17 65.95 69.93
78.86 79.18 83.58 81.00 83.94 81.38 83.94 81.35 86.33 83.61 84.45 82.05 82.90 81.27 86.19 83.61 86.23 84.02 88.02 85.92 93.87 89.62 83.08 80.66 84.54 83.18 84.77 82.14 83.57 80.45 84.14 81.82 83.61 81.67 82.56 81.10
77.49 79.12 76.50 78.59 75.60 77.96 75.61 78.00 78.86 79.59 77.43 78.71 78.74 79.21 71.01 74.36 72.40 75.53 72.53 75.44 53.54 62.84 76.44 78.64 78.23 78.39 76.07 78.41 73.43 75.64 73.25 75.75 77.34 78.54 79.57 80.47
60.89 61.87 66.46 64.38 67.34 65.86 67.24 65.87 63.00 62.54 63.25 62.86 63.05 63.15 75.79 74.00 70.63 69.08 73.22 70.33 91.08 84.52 63.75 62.68 65.46 66.28 69.06 64.77 79.13 76.34 76.04 73.85 66.39 65.90 57.77 59.80
72.43 74.90 70.98 74.62 70.21 74.23 70.22 74.20 73.12 74.33 71.05 74.23 72.10 74.76 65.88 71.17 66.20 71.03 65.04 69.75 50.75 60.83 71.79 75.09 70.94 73.12 71.03 74.47 69.87 73.34 68.23 72.40 71.16 74.36 72.76 75.20
69.88 70.53 75.02 72.69 75.64 73.62 75.59 73.61 74.66 73.08 73.85 72.45 72.97 72.21 80.99 78.81 78.43 76.55 80.62 78.12 92.47 87.07 73.41 71.67 75.00 74.73 76.91 73.45 81.35 78.40 80.09 77.83 75.00 73.78 70.17 70.45
87.07 90.38 90.29 93.03 89.84 92.84 89.85 92.92 87.82 89.32 94.03 94.31 96.06 95.24 89.90 92.29 76.68 86.79 76.52 85.20 37.52 65.20 80.61 88.68 90.32 91.45 84.02 89.87 94.26 95.06 92.79 94.28 93.54 93.99 91.61 93.28
45.88 41.16 32.00 28.80 32.35 29.32 32.30 29.15 41.85 38.84 23.69 23.32 19.25 20.65 42.33 34.20 47.18 41.81 62.61 50.87 95.79 82.82 47.64 40.92 35.77 34.20 50.64 41.37 27.98 24.57 33.03 26.54 26.52 25.65 28.32 27.65
80.45 84.93 87.69 90.15 88.38 90.18 88.37 90.30 83.22 85.09 92.68 92.48 96.19 94.70 92.30 92.59 93.08 93.25 96.26 95.91 61.41 74.20 73.67 81.68 97.32 96.52 72.58 80.50 93.01 93.87 93.09 93.82 94.53 93.65 91.36 92.28
63.02 53.25 46.96 39.36 46.90 39.24 46.90 38.89 55.41 48.64 31.56 29.42 20.52 22.14 43.08 35.08 27.88 27.59 21.81 19.04 92.42 78.16 70.87 58.03 14.02 17.21 82.81 68.07 38.85 32.87 36.95 32.42 29.72 26.78 33.98 31.02
81.47 84.20 86.32 88.10 86.37 87.96 86.37 88.07 83.72 84.87 89.28 89.11 90.59 89.54 85.57 86.84 82.74 86.13 80.91 84.37 50.81 65.86 77.61 82.66 88.24 88.01 77.53 82.42 85.68 87.00 86.19 87.59 90.00 89.50 88.04 88.94
56.50 53.87 43.40 41.69 43.60 42.03 43.49 41.83 54.35 51.13 38.76 40.01 39.64 41.50 52.88 49.94 45.92 44.21 62.62 55.34 93.86 82.84 60.15 54.24 45.55 45.31 67.45 58.03 62.61 58.59 54.61 50.91 38.09 38.93 37.35 38.30
83.00 86.50 88.10 90.43 88.20 90.33 88.20 90.43 84.92 86.43 92.00 91.97 94.28 93.16 89.26 90.57 84.17 88.72 84.56 88.49 49.91 68.42 77.30 84.34 91.96 91.99 78.04 84.26 90.98 91.98 90.69 91.90 92.69 92.38 90.34 91.50
55.13 49.43 40.79 36.62 40.95 36.86 40.90 36.62 50.54 46.20 31.34 30.92 26.47 28.10 46.10 39.74 40.33 37.87 49.01 41.75 94.02 81.27 59.55 51.06 31.78 32.24 66.97 55.82 43.15 38.68 41.53 36.62 31.44 30.45 33.22 32.32
Table 11. ResNet-18 — per-dataset AUROC / FPR95, standalone vs. PLUS. SSB-hard
Method
NINCO
Near-OOD Average
iNaturalist
Textures
OpenImage-O
Far-OOD Average
AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ MSP MSP + PLUS MaxLogit MaxLogit + PLUS Energy Energy + PLUS GEN GEN + PLUS KL-Matching KL-Matching + PLUS ReAct ReAct + PLUS ASH ASH + PLUS DICE DICE + PLUS ViM ViM + PLUS KNN KNN + PLUS MDS MDS + PLUS RMDS RMDS + PLUS MDS++ MDS++ + PLUS RMDS++ RMDS++ + PLUS GradNorm GradNorm + PLUS SHE SHE + PLUS NNGuide NNGuide + PLUS fDBD fDBD + PLUS
74.08 75.57 74.97 77.67 74.69 77.60 74.59 77.36 71.91 73.49 75.45 77.73 77.69 79.65 75.17 78.19 65.46 70.94 64.68 70.03 49.38 61.30 71.42 74.48 70.14 72.73 72.40 74.85 68.32 75.30 74.50 78.25 77.99 79.34 72.34 74.87
73.95 76.54 76.27 74.00 76.46 74.24 75.65 73.64 84.26 82.02 74.63 73.11 70.64 69.64 76.64 74.32 80.61 79.26 85.89 84.04 90.94 87.64 79.21 77.64 84.39 82.68 80.77 79.79 79.63 76.47 74.55 72.72 72.56 71.46 75.04 75.65
82.04 83.41 82.41 84.50 81.76 84.14 83.69 85.09 82.48 83.46 81.18 83.27 81.67 83.78 79.93 83.23 80.63 82.73 81.30 83.07 65.53 73.20 84.03 84.96 85.49 85.26 85.03 85.45 60.14 71.15 72.42 78.94 83.46 84.58 83.64 84.81
56.71 57.44 59.49 53.65 59.76 54.55 54.72 49.52 65.32 58.32 61.05 56.70 64.04 60.14 63.88 58.36 55.65 54.55 58.78 57.52 78.58 72.30 49.03 48.51 52.75 52.60 51.84 50.16 85.29 80.84 72.82 69.87 58.99 55.96 49.30 49.28
78.06 79.49 78.69 81.09 78.22 80.87 79.14 81.22 77.19 78.47 78.31 80.50 79.68 81.72 77.55 80.71 73.04 76.84 72.99 76.55 57.45 67.25 77.72 79.72 77.81 79.00 78.72 80.15 64.23 73.22 73.46 78.59 80.72 81.96 77.99 79.84
65.33 66.99 67.88 63.83 68.11 64.39 65.19 61.58 74.79 70.17 67.84 64.91 67.34 64.89 70.26 66.34 68.13 66.91 72.34 70.78 84.76 79.97 64.12 63.08 68.57 67.64 66.31 64.97 82.46 78.66 73.69 71.30 65.78 63.71 62.17 62.47
89.95 92.39 91.27 94.30 90.39 94.14 93.24 94.93 90.91 91.92 93.30 94.86 94.62 95.72 89.97 94.06 92.03 94.72 91.68 94.41 77.34 88.26 91.11 94.12 95.79 95.76 94.80 95.38 72.24 87.05 84.40 91.74 95.53 95.71 93.74 95.33
39.03 33.88 31.10 24.23 31.46 24.19 24.49 20.88 36.28 31.95 24.78 21.52 25.26 20.55 35.06 25.57 24.66 20.53 29.85 21.98 54.02 40.35 24.02 22.08 16.69 17.82 21.07 19.16 70.78 56.62 48.36 37.43 21.65 18.36 20.94 19.38
82.87 86.28 86.04 89.27 85.89 89.16 87.07 89.59 85.27 87.13 88.18 90.36 93.72 93.92 86.63 89.62 97.49 97.04 97.03 96.51 91.41 93.12 90.94 91.90 97.21 96.36 87.42 89.66 68.53 79.96 78.66 85.95 91.90 92.12 92.45 93.14
58.67 48.95 54.77 41.83 54.76 41.54 48.90 38.81 61.09 47.27 47.35 38.87 31.32 28.11 55.35 43.46 11.88 14.10 15.56 15.23 35.98 31.37 34.76 32.33 13.88 16.51 43.76 37.62 79.46 71.53 65.85 55.71 41.23 34.39 28.69 26.72
87.02 88.88 89.75 91.26 89.42 91.03 90.63 91.56 88.10 89.05 90.33 91.02 91.31 91.85 88.16 90.19 90.79 92.14 90.94 92.04 74.96 83.19 88.40 90.35 93.26 92.71 90.21 91.10 67.18 78.12 78.04 84.59 91.66 91.50 91.46 92.41
45.55 41.79 37.74 33.81 38.56 34.30 32.46 30.56 44.13 39.46 38.75 36.57 40.71 36.45 44.64 39.76 30.52 28.94 34.18 30.36 62.20 52.22 30.83 31.45 26.72 28.09 32.30 31.46 83.84 78.37 67.72 60.47 39.25 36.20 27.95 27.61
Table 12. Wide-ResNet50-2 — per-dataset AUROC / FPR95, standalone vs. PLUS.
15
86.61 89.18 89.02 91.61 88.57 91.44 90.31 92.03 88.09 89.37 90.60 92.08 93.22 93.83 88.25 91.29 93.44 94.63 93.22 94.32 81.24 88.19 90.15 92.12 95.42 94.94 90.81 92.05 69.32 81.71 80.37 87.43 93.03 93.11 92.55 93.63
47.75 41.54 41.20 33.29 41.59 33.34 35.28 30.08 47.17 39.56 36.96 32.32 32.43 28.37 45.02 36.26 22.35 21.19 26.53 22.52 50.73 41.31 29.87 28.62 19.10 20.81 32.38 29.41 78.03 68.84 60.64 51.20 34.04 29.65 25.86 24.57
Method
SSB-hard
NINCO
Near-OOD Average
iNaturalist
Textures
OpenImage-O
Far-OOD Average
AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ MSP MSP + PLUS MaxLogit MaxLogit + PLUS Energy Energy + PLUS GEN GEN + PLUS KL-Matching KL-Matching + PLUS ReAct ReAct + PLUS ASH ASH + PLUS DICE DICE + PLUS ViM ViM + PLUS KNN KNN + PLUS MDS MDS + PLUS RMDS RMDS + PLUS MDS++ MDS++ + PLUS RMDS++ RMDS++ + PLUS GradNorm GradNorm + PLUS SHE SHE + PLUS NNGuide NNGuide + PLUS fDBD fDBD + PLUS
75.47 79.39 72.93 78.49 65.59 76.57 79.40 81.32 72.84 75.93 73.81 78.85 57.65 69.60 48.03 65.63 71.67 76.52 72.27 76.86 74.15 77.81 75.83 78.70 75.85 78.32 76.90 79.04 46.44 66.29 71.90 76.99 43.32 62.34 73.27 77.34
76.75 68.19 85.60 75.97 90.04 81.71 67.32 62.55 81.16 74.27 76.74 70.96 90.67 84.93 96.98 90.57 87.06 78.88 78.06 73.90 77.26 69.79 74.95 68.10 76.39 72.12 74.29 70.38 96.25 94.74 85.07 81.50 97.09 93.88 76.52 72.04
81.08 85.81 76.43 83.96 64.05 80.32 86.51 88.59 82.00 84.64 81.29 86.77 65.90 80.34 36.35 61.33 84.53 87.46 83.56 87.09 85.12 87.48 86.06 88.08 87.13 88.51 87.77 88.87 34.65 59.08 77.58 83.88 38.10 63.41 84.35 87.28
71.43 52.05 86.40 66.65 92.18 79.24 50.71 42.70 65.01 53.03 62.77 48.53 77.65 63.65 99.92 97.43 56.46 43.83 56.03 48.69 52.14 44.14 48.07 42.58 48.15 43.16 46.20 42.56 97.83 96.85 81.56 69.90 98.94 94.78 54.36 47.48
78.28 82.60 74.68 81.22 64.82 78.44 82.96 84.95 77.42 80.28 77.55 82.81 61.78 74.97 42.19 63.48 78.10 81.99 77.91 81.97 79.64 82.65 80.94 83.39 81.49 83.41 82.34 83.96 40.55 62.69 74.74 80.44 40.71 62.88 78.81 82.31
74.09 60.12 86.00 71.31 91.11 80.47 59.02 52.62 73.09 63.65 69.75 59.74 84.16 74.29 98.45 94.00 71.76 61.35 67.05 61.30 64.70 56.97 61.51 55.34 62.27 57.64 60.25 56.47 97.04 95.79 83.31 75.70 98.02 94.33 65.44 59.76
89.54 93.89 82.63 91.67 55.75 86.77 95.36 96.48 91.15 93.05 87.87 94.55 58.76 85.17 11.47 51.31 94.85 96.36 92.72 95.72 94.20 95.79 94.35 95.88 95.55 96.31 96.32 96.56 18.74 61.46 85.18 91.51 22.48 66.64 94.35 96.11
43.46 28.11 77.82 40.36 92.24 66.15 19.23 15.94 35.78 26.72 36.76 22.59 83.40 55.09 100.00 99.96 15.56 13.65 22.32 17.39 19.82 16.37 21.44 17.57 17.34 16.45 16.30 15.56 97.96 95.70 78.10 54.09 99.56 94.85 19.91 16.15
74.36 81.08 64.66 74.91 49.48 68.54 83.63 86.39 87.69 88.96 76.30 82.77 72.80 83.94 75.53 86.35 93.47 93.61 92.23 92.33 91.48 91.92 90.62 91.33 92.09 91.92 91.29 91.23 12.08 32.34 76.97 83.60 57.96 75.65 90.33 90.91
96.46 82.38 98.11 95.74 98.27 97.10 86.15 67.10 44.95 37.06 91.24 80.35 75.69 63.02 92.96 58.46 26.83 26.97 31.70 29.94 32.85 31.60 36.84 33.94 32.49 31.18 35.12 33.91 99.07 98.77 83.84 75.33 97.98 85.03 38.93 35.75
83.10 88.66 73.40 83.87 52.68 76.44 90.92 92.75 89.26 91.07 82.94 89.64 68.35 84.92 43.50 69.76 93.33 94.47 92.19 93.87 92.92 94.06 93.00 94.08 93.77 94.18 94.03 94.21 15.91 42.80 78.17 86.33 41.61 68.92 92.53 93.82
74.66 45.27 93.53 75.39 96.42 90.05 36.82 28.41 43.08 32.50 58.70 38.22 74.84 52.26 99.92 95.53 22.80 20.33 27.21 22.13 25.92 22.28 27.01 22.92 23.22 21.23 23.40 21.40 98.88 98.39 84.95 69.84 99.30 95.08 27.84 22.93
82.33 87.88 73.56 83.48 52.64 77.25 89.97 91.87 89.37 91.03 82.37 88.99 66.64 84.68 43.50 69.14 93.88 94.81 92.38 93.97 92.87 93.92 92.66 93.76 93.80 94.14 93.88 94.00 15.58 45.53 80.11 87.15 40.68 70.40 92.40 93.61
71.53 51.92 89.82 70.50 95.64 84.43 47.40 37.15 41.27 32.09 62.23 47.05 77.98 56.79 97.63 84.65 21.73 20.32 27.08 23.15 26.20 23.42 28.43 24.81 24.35 22.95 24.94 23.62 98.64 97.62 82.30 66.42 98.95 91.65 28.89 24.94
Table 13. ConvNeXt-B — per-dataset AUROC / FPR95, standalone vs. PLUS.
Method
SSB-hard
NINCO
Near-OOD Average
iNaturalist
Textures
OpenImage-O
Far-OOD Average
AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ AUROC↑ FPR95↓ MSP MSP + PLUS MaxLogit MaxLogit + PLUS Energy Energy + PLUS GEN GEN + PLUS KL-Matching KL-Matching + PLUS ReAct ReAct + PLUS ASH ASH + PLUS DICE DICE + PLUS ViM ViM + PLUS KNN KNN + PLUS MDS MDS + PLUS RMDS RMDS + PLUS MDS++ MDS++ + PLUS RMDS++ RMDS++ + PLUS GradNorm GradNorm + PLUS SHE SHE + PLUS NNGuide NNGuide + PLUS fDBD fDBD + PLUS
70.99 74.81 68.16 73.21 63.88 71.62 73.21 75.66 70.17 72.68 62.88 71.44 49.03 62.10 60.41 70.94 71.55 74.44 72.18 74.78 73.26 75.28 73.95 75.68 74.87 76.12 74.55 76.01 46.21 63.49 75.70 77.57 64.72 72.41 66.27 71.74
84.39 79.02 87.17 83.52 88.85 85.36 79.40 77.96 85.96 83.36 89.49 85.92 93.80 90.06 88.60 82.54 87.68 85.22 81.77 80.82 80.84 79.54 81.96 79.67 80.35 79.65 81.98 80.54 95.52 94.18 80.45 78.20 82.38 80.06 86.98 83.40
80.86 84.19 75.89 81.57 68.31 78.36 85.61 87.01 82.40 84.26 65.21 76.90 54.49 68.78 56.26 72.04 84.13 86.27 84.70 86.43 85.42 86.73 86.30 87.25 87.04 87.56 86.88 87.53 38.01 60.73 83.23 85.88 67.11 78.59 79.65 83.70
64.09 52.96 85.96 74.80 91.32 83.86 50.30 46.96 63.78 52.87 93.12 85.97 93.50 87.28 96.03 87.81 64.82 52.77 53.25 49.84 50.26 46.86 47.62 45.26 47.79 45.24 48.61 45.82 97.01 95.92 70.26 61.12 80.57 69.56 69.70 60.85
75.92 79.50 72.03 77.39 66.09 74.99 79.41 81.34 76.28 78.47 64.05 74.17 51.76 65.44 58.33 71.49 77.84 80.35 78.44 80.61 79.34 81.00 80.12 81.47 80.96 81.84 80.72 81.77 42.11 62.11 79.47 81.72 65.91 75.50 72.96 77.72
74.24 65.99 86.56 79.16 90.08 84.61 64.85 62.46 74.87 68.11 91.31 85.94 93.65 88.67 92.31 85.18 76.25 69.00 67.51 65.33 65.55 63.20 64.79 62.47 64.07 62.45 65.30 63.18 96.27 95.05 75.36 69.66 81.47 74.81 78.34 72.12
88.76 92.37 90.06 93.51 88.16 93.52 94.24 95.31 89.50 91.40 86.95 93.30 37.61 68.00 81.46 92.04 95.42 96.10 93.85 95.04 93.57 94.88 93.85 94.97 95.60 95.71 94.78 95.19 39.59 77.02 96.22 96.33 88.42 93.91 91.01 93.88
40.06 31.75 47.00 31.39 55.86 33.09 24.14 21.10 45.14 33.82 59.12 34.13 97.46 91.77 64.59 40.01 15.34 15.47 25.94 22.09 23.38 20.69 23.72 20.90 19.83 18.72 22.22 20.43 93.60 88.36 19.19 17.09 43.93 30.19 33.78 25.91
85.51 87.12 85.87 87.62 83.65 86.83 90.15 90.17 86.66 87.54 82.55 86.37 44.82 63.88 76.02 83.24 92.74 92.50 92.03 91.50 89.49 89.84 89.52 89.78 92.03 91.28 90.29 90.06 45.51 65.86 91.58 91.53 83.14 86.70 87.94 88.79
54.02 49.41 69.98 61.35 76.52 66.94 44.30 41.80 57.99 49.34 76.68 68.43 92.86 86.65 80.44 70.28 32.25 31.87 36.78 36.32 37.62 38.57 37.69 39.20 35.02 36.54 37.70 39.30 94.39 92.99 47.47 41.92 66.50 59.98 51.60 46.60
85.85 88.69 82.06 86.59 75.68 83.74 91.39 92.07 87.97 89.30 73.24 82.57 45.98 66.51 63.58 77.28 93.24 93.57 92.43 92.70 92.22 92.73 92.40 92.75 93.53 93.12 93.12 92.84 28.73 54.01 88.29 90.19 77.56 85.45 90.07 91.53
49.84 40.66 81.01 64.50 88.92 75.92 33.97 31.71 49.79 40.66 90.55 79.69 96.10 89.34 94.60 83.69 24.97 24.52 30.56 29.31 28.70 27.69 28.66 28.03 27.09 26.80 27.57 27.18 97.87 97.34 59.49 48.33 73.90 59.53 40.34 33.47
Table 14. MaxViT-T — per-dataset AUROC / FPR95, standalone vs. PLUS.
16
86.71 89.39 86.00 89.24 82.50 88.03 91.93 92.52 88.04 89.41 80.91 87.41 42.80 66.13 73.69 84.19 93.80 94.06 92.77 93.08 91.76 92.48 91.92 92.50 93.72 93.37 92.73 92.70 37.94 65.63 92.03 92.68 83.04 88.69 89.67 91.40
47.97 40.61 66.00 52.41 73.77 58.65 34.14 31.54 50.97 41.27 75.45 60.75 95.47 89.25 79.88 64.66 24.19 23.95 31.09 29.24 29.90 28.98 30.02 29.38 27.31 27.35 29.16 28.97 95.29 92.90 42.05 35.78 61.44 49.90 41.91 35.33
D. Why Construct the Subspace from SAE Neurons? Our per-class subspace is spanned by each class’s top-active SAE neurons (Section 3.2). Table 15 extends the comparison of Section 6.1 to all seven backbones: random subspaces yield essentially no gain over MDS++, and all SAE-free alternatives suffer larger worst-case shortfalls. Beyond this consistency, the SAE provides an analytical advantage over per-class PCA, detailed below. Subspace (m=16) MDS++ alone (no z) SAE top-active (ours) Per-class PCA Global PCA + per-class Σc Top-16 raw channels Random SAE neurons† Random orthonormal†
ViT-B/16
ResNet-50
Swin-T
ResNet-18
Wide-ResNet50-2
ConvNeXt-B
MaxViT-T
Mean
Max gap
79.87 81.23 81.64 80.05 79.82 79.34 79.57
74.71 78.22 76.69 77.01 76.75 74.83 74.47
78.92 80.90 81.42 79.41 78.76 78.00 78.49
70.94 73.30 71.96 73.87 72.28 71.37 71.16
77.81 79.03 79.96 79.77 78.69 77.59 77.59
81.49 83.53 83.73 82.86 81.90 81.42 81.49
80.96 81.90 82.48 81.40 81.23 81.05 81.10
77.81 79.73 79.70 79.20 78.49 77.66 77.70
– 0.93 1.91 2.01 2.66 3.42 3.75
Table 15. Choice of the 16-dimensional per-class subspace on all seven backbones (near-OOD AUROC of MDS++ + 0.5 z, %). Every row uses the per-class metric Σc and differs only in the subspace. Global PCA: one 16-dimensional PCA basis shared by all classes; top-16 raw channels: each class’s 16 feature coordinates with the largest mean value. Max gap: the largest shortfall, over the seven backbones, from the best subspace on that backbone (lower is better). Bold: best among the subspaces. † Mean over 5 seeds (std ≤ 0.15). On far-OOD, all non-random subspaces stay within 0.82 points of the best on every backbone. All rows are re-estimated in the pipeline of Table 4.
The SAE top-active neurons provide consistent stability. No single subspace is best on every backbone (Table 15): perclass PCA leads on five of the seven backbones, global PCA on ResNet-18, and the SAE top-active neurons on ResNet-50. The SAE subspace, however, never falls far behind: its largest gap to the best subspace on any backbone is 0.93 fused nearAUROC points (Wide-ResNet50-2), versus 1.91 for per-class PCA, 2.01 for global PCA, and 2.66 for raw channels, while its seven-backbone mean is on par with per-class PCA (79.73 vs. 79.70). Thus, while the subspace spanned by the SAE topactive neurons is the best 16-dimensional subspace on only one of the seven backbones, it avoids the sharp near-OOD drops that every alternative incurs on some backbone. For a detector-agnostic signal designed to complement diverse backbones, we value this consistency over the backbone-specific gains it forgoes. A shared basis for cross-class analysis. Each class’s subspace is spanned by its top-active neurons, which represent the most characteristic directions of that class, selected from the global dictionary of a shared SAE. Because every class selects from the same dictionary, the characteristic neurons of different classes overlap explicitly and traceably; per-class PCA instead assigns each class a distinct orthonormal basis with no shared elements, precluding cross-class correspondence. This overlap is semantically structured: related classes reuse the same characteristic neurons substantially more than unrelated ones (Table 16). Across the three main backbones, the 118 ImageNet dog breeds reuse the same characteristic neurons at 2.7–8.9× the rate of a size-matched random class group, and each class shares neurons with its nearest neighbor by feature centroid at 9–85× the random-pair rate. The SAE top-active neurons thus provide shared axes that recur across related classes, establishing a basis for cross-class analysis that a per-class PCA basis does not afford. Backbone
Random pair
Nearest class
Dogs (118)
Snakes (17)
Birds (18)
ResNet-50 Swin-T ViT-B/16
0.032 0.011 0.003
0.296 (9×) 0.277 (26×) 0.264 (85×)
0.087 (2.7×) 0.029 (2.7×) 0.031 (8.9×)
0.154 (4.6×) 0.160 (15.4×) 0.211 (64.9×)
0.091 (2.7×) 0.126 (11.1×) 0.094 (23.1×)
Table 16. Characteristic-neuron reuse across classes (mean pairwise top-16 support Jaccard). Random pair: two unrelated classes. Nearest class: each class and its closest class by feature centroid. Dogs/Snakes/Birds: mean within-group Jaccard for the ImageNet dog (151–268), snake (52–68), and bird (7–24) groups; the parenthetical is the ratio to a size-matched random group of classes (× vs. the random pair for the nearest-class column).
We do not claim that the SAE top-active neurons form a performance-optimal subspace. Rather, weighing this uniform crossbackbone stability and the shared, analyzable structure against a fused near-OOD cost of at most 0.93 points, we propose them as a deliberate methodological choice for constructing the per-class subspace.
17
Related documents
- Optimizing Three Critical Factors for Practical and Effective OOD Detection Fine-Tuning#997898
- Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning#1355560
- Tracking the Discriminative Axis: Dual Prototypes for Test-Time OOD Detection Under Covariate Shift#1488322
- General OOD detection via model-aware and subspace-aware variable priority.#621035
- Unified Multi-Layer Subspace Modeling for Cross-Domain OOD Detection#974873
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive