Diffusion-Guided Feature Selection via Nishimori Temperature: Noise-Based Spectral Embedding

arXiv:2604.24692v1 [cs.LG] 27 Apr 2026

Diffusion-Guided Feature Selection via Nishimori Temperature: Noise-Based Spectral Embedding Vasiliy S. Usatyuk, Denis A. Sapozhnikov

Sergey I. Egorov

R&D Department T8 LLC Moscow, Russia Email: [email protected], [email protected]

Department of Computer Science South-West State University Kursk, Russia Email: [email protected]

Abstract—We introduce Noise-Based Spectral Embedding (NBSE), a diffusion-driven framework for selecting informative features from high-dimensional data. Given M objects and D original features, NBSE constructs a sparse similarity graph on the objects and determines the Nishimori temperature βN — the critical inverse temperature at which the Bethe–Hessian matrix becomes singular. The associated smallest eigenvector identifies the dominant mode of a degree-corrected diffusion process whose time scale is set automatically by statistical physics. Repeating this construction for every original feature yields a Ddimensional spectral fingerprint. By transposing the data matrix and applying NBSE to the features themselves, we obtain a low-dimensional representation of the feature space that reveals natural groups of redundant or semantically related dimensions. Selecting one representative per group provides a principled, non-greedy dimensionality reduction that preserves classification performance while discarding up to 70% of the original features. The Bethe–Hessian operator H(β) = I + D̃(β) − S(β) incorporates a degree-dependent diagonal term D̃ii (β) = P 2 j sinh (βWij ) that is essential for sparse graphs. We show that this term yields an intrinsic degree correction which mitigates hub dominance in the spectrum, and we prove a perturbation bound establishing that coloured Gaussian noise of local variance σi2 shifts βN by at most O(σ̄ 2 ), with σ̄ = maxi σi , thereby rigorously justifying the method’s robustness. Experiments on deep-network embeddings (MobileNetV2 and EfficientNet-B4) demonstrate that NBSE consistently outperforms classical ANOVA F -test selection and random sampling. On MobileNetV2, spectral selection exhibits high stability, with accuracy fluctuations bounded by 5% under aggressive feature reduction, whereas original-data methods lose up to 25%. On EfficientNet-B4, the synergy of Nishimori-temperature spectral embedding and specialised selection limits accuracy loss to less than 1% even at 70% feature reduction (30% retention), outperforming baseline methods by up to 6.8%. Index Terms—Diffusion on graphs, Nishimori temperature, Bethe–Hessian, noise-based spectral embedding, feature selection, MET QC-LDPC graphs, dimensionality reduction.

I. I NTRODUCTION High-dimensional representations—deep-network embeddings, genomic vectors, TF-IDF matrices—are now standard in modern machine learning. While such rich descriptors enable state-of-the-art accuracy, they create three practical difficulties: • Overfitting. Many dimensions capture noise or irrelevant variation. • Computational burden. Training and inference scale at least linearly with the number of features.

•

Interpretability loss. Tracing a decision back to the original measurements becomes increasingly hard.

Classical dimensionality-reduction techniques such as PCA [1] or manifold learners (t-SNE [2], UMAP [3]) focus on preserving the geometry of the object space. When downstream models are linear, however, many features are redundant because they convey essentially the same information with respect to an underlying similarity graph. A widely used remedy is the ANOVA F -test [4], which scores each feature independently and discards those with low marginal relevance. This filter approach ignores inter-feature correlations; consequently it may remove useful dimensions or retain noisy ones, especially when features are strongly collinear. In this paper we propose Noise-Based Spectral Embedding (NBSE)— a topology-aware, diffusion-guided method that: 1) constructs a sparse quasi-cyclic similarity multigraph on the objects, 2) computes the Nishimori temperature βN , i.e. the critical inverse temperature at which the Bethe–Hessian H(β) becomes singular (Eq. (3)), 3) extracts the eigenvector associated with the smallest eigenvalue of H(βN ), which identifies the dominant mode of a degree-corrected diffusion process evaluated at the statistically optimal inverse temperature, 4) repeats this procedure for every original feature, thereby obtaining a spectral fingerprint that quantifies per-feature participation in graph diffusion, and 5) embeds features themselves by transposing the data matrix and applying NBSE to the columns, enabling spectral ablation: a principled reduction that respects global correlations without greedy search. Because the optimal inverse temperature is obtained analytically from the Nishimori condition rather than tuned via crossvalidation, NBSE remains computationally lean: under a quasistationarity hypothesis, a single sparse eigenvalue problem suffices for all features. The rest of the paper is organised as follows. Section II reviews related work; Section III introduces the Ising model, the Nishimori temperature, and the Bethe–Hessian, derives the degree-corrected diffusion interpretation, and proves a

noise-robustness bound; Section IV details NBSE and the spectral ablation algorithm; Section V presents experimental results; Section VI discusses why degree-corrected diffusion provides a superior basis for feature selection; and Section VII concludes. II. R ELATED W ORK Dimensionality reduction for classification has a long tradition. Linear methods such as LDA [5] and PCA maximise class separation or variance, respectively, but do not directly address feature redundancy. Non-linear manifold learners (tSNE, UMAP) preserve local neighbourhoods of objects yet are unsuitable for selecting a subset of original features. Feature-selection techniques fall into three families: • Filter methods score each dimension independently using statistical tests such as the ANOVA F -test or mutual information [4]. • Wrapper methods rely on greedy forward/backward selection with a classifier in the loop [6]. • Embedded methods perform selection implicitly via regularisation (Lasso, Elastic Net) [7]. Spectral techniques have been employed for clustering [8] and dimensionality reduction through Laplacian eigenmaps [9], but they rarely target feature selection directly. Graph-based diffusion for semi-supervised learning [10] inspired our use of a diffusion operator to reveal feature importance. From the statistical-physics perspective, Saade et al. [15] introduced spectral clustering based on the Bethe–Hessian matrix and showed that its smallest eigenvalue vanishes at the Nishimori temperature—the phase-transition point of an Ising model with quenched disorder. Dall’Amico et al. [11] unified this framework for sparse graphs, proving that the Bethe– Hessian outperforms both adjacency and standard Laplacian methods precisely because it incorporates a degree-dependent diagonal correction. This insight has been leveraged for node classification in sparse graphs [12], [13], but never for feature selection. Our method builds on these ideas: we construct a multiedge quasi-cyclic LDPC similarity graph (high sparsity, known spectral properties), compute the Nishimori temperature βN as the root of λmin (H(β)) = 0, and use the associated eigenvector for diffusion-style feature ranking. The key innovation is that we apply this machinery twice—once on objects (to build diffusion fingerprints) and once on features (to perform spectral ablation)—thereby exploiting global interfeature correlations that filter methods miss. III. BACKGROUND A. Ising model, Nishimori temperature and Bethe–Hessian Let G = (V, E, W) be a weighted undirected graph with |V | = M objects and symmetric adjacency matrix W ∈ RM ×M . The Ising Hamiltonian on G reads X H(s) = − Wij si sj , si ∈ {−1, +1}. (1) i<j

At inverse temperature β, the Boltzmann distribution is pβ (s) =

e−βH(s) . Z(β)

(2)

The Nishimori line in parameter space corresponds to the set of points where the temperature matches the noise level of a planted spin-glass model [14]. At the special Nishimori temperature βN , the model undergoes a phase transition from a paramagnetic (disordered) phase to an ordered phase in which cluster structure becomes visible. This critical point is characterised by the singularity condition
λmin H(βN ) = 0, (3) where H(β) denotes the Bethe–Hessian matrix [11], [15]. Defining effective edge weights ωij (β) = tanh(βWij ), its entries are  X ω 2 (β)  ωij (β) ik − (4) Hij (β) = δij 1 + 2 2 (β) . 1 − ωik (β) 1 − ωij k∈∂i

Equivalently, using hyperbolic identities, H(β) = I + D̃(β) − S(β), X D̃ii (β) = sinh2 (βWij ),

(5) (6)

j

  sinh(βWij ) cosh(βWij ) Sij (β) = = 12 sinh(2βWij ), (i, j) ∈ E,   0, otherwise.

(7)

The diagonal matrix D̃(β) encodes a local effective-degree term: each entry aggregates the thermal response sinh2 (βWij ) of edges incident to node i. Because D̃ii (β) depends on both the inverse temperature and the local weight distribution, it varies from node to node on heterogeneous graphs. This builtin degree correction prevents high-degree hub nodes from dominating the spectrum—a well-known failure mode of both the adjacency matrix and the standard unnormalised Laplacian on sparse graphs [11], [15]. The eigenvector ψ min associated with the smallest (near-zero) eigenvalue of H(βN ) provides a spectral coordinate that separates nodes belonging to different communities. Remark 1 (Nishimori identities). On the Nishimori line β = βN , exact gauge-symmetry identities hold [14]. In particular, thermal averages of local observables satisfy relations such as ⟨si sj ⟩βN = tanh(βN Wij ) and the internal energy per bond is self-averaging. These identities underlie the noise-robustness proof in Section III-C. B. Diffusion interpretation Consider the standard P combinatorial Laplacian L = Dg − W, where (Dg )ii = j Wij is the weighted graph degree, and its heat kernel K(t) = exp(−tL).

(8)

For any initial signal f (0), diffusion evolves as f (t) = K(t) f (0). Expanding K(t) in the eigenbasis of L yields X K(t) = e−tλk uk u⊤ k, k

so that for large t the mode associated with the smallest nonzero eigenvalue of L dominates. The Bethe–Hessian defines its own degree-corrected diffusion family. At β = 0, D̃(0) = 0 and S(0) = 0, so that H(0) = I,

(9)

which corresponds to the stable paramagnetic state with a uniform restoring force. As β increases, the off-diagonal term −S(β) introduces graph structure while the diagonal I+ D̃(β) counteracts it with a node-dependent restoring force. To make the relationship precise, define the Bethe–Hessian Laplacian LBH (β) := I − (I + D̃)−1/2 S(β) (I + D̃)−1/2 ,

(10)

which is well defined for all β ≥ 0 because I + D̃ is strictly positive definite. Because H(β) and LBH (β) are related by a congruence transformation, they share the same inertia; in particular, at βN the null space of LBH (βN ) is the image under (I + D̃)1/2 of the null space of H(βN ). Consequently, LBH identifies exactly the same cluster structure. The operator LBH acts as a generalised Laplacian whose random-walk transition probabilities are reweighted by effective couplings Sij (β) and normalised by local susceptibilities (1 + D̃ii ). This weighting suppresses edges that exceed the thermal scale (βWij ≫ 1), preventing dominant edges from distorting the diffusion. In physical terms, computing the Nishimori temperature is equivalent to finding the optimal inverse diffusion parameter at which the slowest nontrivial degree-corrected mode becomes statistically distinguishable from noise. The following proposition characterises the small-β expansion of H(β) and makes explicit why D̃(β) cannot be replaced by a scalar multiple of the identity on heterogeneous graphs. Proposition 1 (Small-β expansion). For β → 0, the Bethe– Hessian admits the expansion P M H(β) = I − β W + β 2 diag j Wij2 + O(β 3 ). (11) i=1

3

Proof: Since sinh(x) = x+x /6+O(x5 ) and cosh(x) = 1 + x2 /2 + O(x4 ), we have sinh2 (x) = x2 + O(x4 ) and sinh(x) cosh(x) = x + 23 x3 + O(x5 ). Substituting x = βWij into the definitions of D̃(β) and S(β) yields the stated expansion.

weighted) nodes receive larger diagonal entries, which reduces their influence in the spectrum of H(β) and removes the spectral bias that plagues adjacency-based methods. Conflating D̃(β) with a scalar multiple of I would destroy this intrinsic degree correction. C. Noise model for robustness To study measurement noise we perturb each object vector with coloured Gaussian noise:
e i = pi + εi , p εi ∼ N 0, σi2 Id . (12) The local scale σi is proportional to the typical distance from node i to its k-nearest neighbours, ensuring that the perturbation is small relative to inter-cluster gaps. Adding this noise before graph construction perturbs the edge weights. Provided the similarity function is smooth and the signal-to-noise ratio is bounded, the resulting relative change in each weight satisfies a bound of the form fij − Wij | ≤ ε|Wij | with ε = O(σ̄ 2 ), where σ̄ = maxi σi . |W We now prove that, provided this relative perturbation is small and the smallest eigenvalue curve of H(β) is non-degenerate at its root, the Nishimori temperature shifts only at first order in the weight error, and hence at second order in the noise amplitude. Proposition 2 (Stability of the Nishimori temperature under noise). Let G be a sparse graph with bounded average degree, Bethe–Hessian H(β; W) and Nishimori temperature βN . Let f denote the edge weights obtained after perturbing the W data according to Eq. (12), and let βeN be the corresponding Nishimori temperature. Suppose there exists a constant ε > 0 such that for every edge (i, j) ∈ E, fij − Wij | ≤ ε |Wij |. |W

(13)

If the non-degeneracy condition g := ∂β λmin (H(β)) β=β ̸= N 0 holds, then p C1 ε |E| e + O(ε2 ) (14) | βN − β N | ≤ |g| for a constant C1 depending on βN and the weight scale. In particular, for coloured noise with local variance σi2 , we have ε = O(σ̄ 2 ) where σ̄ = maxi σi . Proof: The proof combines three ingredients. (a) Perturbation of the Bethe–Hessian. From Eq. (4), the fij − Wij has diagonal and change in H induced by δWij = W off-diagonal contributions: X  δHii = sinh2 (β(Wij + δWij )) − sinh2 (βWij ) j

Remark 2 (Role of the degree-dependent diagonal). On a dregular graph with uniform weights Wij = w, every node has P 2 2 2 j Wij = dw and the β term in Eq. (11) is proportional to the identity. However, on sparse real-world graphs—including our QC-LDPC similarity graphs built from P data—effective degrees vary considerably. The term diag( j Wij2 ) then encodes essential local heterogeneity: high-degree (or strongly

=β

X

sinh(2βWij ) δWij + O(ε2 ),

j

 δHij = − sinh(β(Wij + δWij )) cosh(β(Wij + δWij ))  − sinh(βWij ) cosh(βWij ) = −β cosh(2βWij ) δWij + O(ε2 ),

i ̸= j.

Because | sinh | and | cosh | are bounded on the support of the weights, both terms are uniformly O(ε). For a sparse graph with bounded average degree, the number of non-zero entries p in H is Θ(|E|); consequently ∥δH∥F ≤ C ′′ ε |E|. (b) Weyl’s inequality and eigenvalue shift. For symmetric matrices, Weyl’s inequality gives p e − λk (H)| ≤ ∥δH∥2 ≤ ∥δH∥F ≤ C ′′ ε |E|. |λk (H) At β = βN we have λminp (H) = 0, so the perturbed matrix e ≤ C ′′ ε |E|. satisfies |λmin (H)| (c) Implicit function theorem. The Nishimori condition λmin (H(β)) = 0 defines a smooth level set near βN because the non-degeneracy hypothesis guarantees g = ∂β λmin ̸= 0. By the implicit function theorem, δλmin + O(ε2 ). βeN − βN = − g β=βN The numerator is bounded by part (b) and the denominator equals |g| by definition. Combining these estimates yields Eq. (14). Finally, for Gaussian noise satisfying Eq. (12), the relative perturbation of each squared distance is O(σi2 /∥pi − pj ∥2 ); assuming a bounded signal-to-noise ratio gives ε = O(σ̄ 2 ). Remark 3. The bound in Eq. (14) confirms that noise acts as a small perturbation on the critical temperature, not as a systematic bias. Physically, this is because at βN the system sits at a phase transition between paramagnetic and ordered regimes; non-degeneracy of the smallest-eigenvalue curve protects the singular condition against local weight fluctuations. IV. P ROPOSED M ETHOD A. Similarity graph construction All subsequent steps rest on a similarity graph among objects. Given raw vectors xi ∈ RD , we define edge weights via the Gaussian kernel with adaptive local scales:  ∥x − x ∥2  i j Wij = exp − , (15) σi σj where the local scale σi is the average distance to the k = 10 nearest neighbours of object i. We instantiate this kernel on a sparse quasi-cyclic LDPC backbone (Section IV-E). B. Noise-Based Spectral Embedding (NBSE) Given a data matrix X ∈ RM ×D (rows = objects, columns = features), we apply the following pipeline for each feature l, using the univariate observations {xil }M i=1 as scalar node attributes: 1) Graph construction. Build a sparse similarity graph G(l) on the M objects via Eq. (15) with distances computed in the one-dimensional feature space. (l) 2) Nishimori temperature. Solve Eq. (3) for βN . This is a one-dimensional root-finding problem; because f (β) := λmin (H(β)) is smooth and monotonically increasing in the relevant regime, bisection or Brent’s method

converges reliably. The smallest eigenvalue is obtained with a few Lanczos iterations exploiting sparsity. (l) 3) Bethe–Hessian eigenvector. Form H(βN ) via Eq. (4) (l) and extract its smallest eigenvector ψ min . By Proposition 1, the degree-dependent diagonal correctly normalises heterogeneous degrees. (l) 4) One-dimensional embedding. The vector ψ min ∈ RM is a diffusion-based coordinate for feature l, indicating how that feature participates in the slowest degreecorrected flow on its graph. Collecting all coordinates yields the spectral fingerprint matrix  (1) (2) (D)  Ψ = ψ min ψ min · · · ψ min ∈ RM ×D . (16) Each column of Ψ encodes the diffusion behaviour of one original feature. Features that induce similar univariate graph topologies—and hence similar eigenvectors—are likely to be redundant or semantically related. Under the quasi-stationary hypothesis we may compute a single global value βN on an aggregate object graph and reuse it for all univariate Bethe–Hessians, reducing complexity from O(D · M log M ) to O(M log M ). Proposition 2 justifies this simplification: sufficiently small inter-feature perturbations shift βN by at most O(ε/|g|). C. Spectral diffusion-guided feature ablation Directly clustering the columns of Ψ would require pairwise comparisons among D vectors. We avoid this cost by embedding features themselves: transpose the data matrix to treat features as nodes in a D-node graph and apply NBSE to the resulting feature–feature affinity. The eigenvector obtained from this transposed diffusion, denoted ϕmin ∈ RD , orders features according to the similarity of their roles in the joint diffusion geometry. Algorithm 1 implements balanced histogram binning on ϕmin : its range is divided into n equal intervals, and one representative per interval is selected. The resulting index set I ⊂ {1, . . . , D}, with |I| = n, preserves the global diffusion structure while discarding redundant dimensions. The topology of the QC-LDPC graph strongly influences the histogram of feature embeddings (Fig. 1). A unique Nishimori temperature together with a low-energy eigenvector produces well-separated clusters in the transposed feature space, allowing binning ablation to select one representative from each cluster. By contrast, Erdős–Rényi graphs do not satisfy the spectral requirements of diffusion-guided ablation; their spectra fail to reveal informative feature groups [12], [13]. The same limitation holds for non-constrained expanders, fully-connected graphs, and line graphs. D. Classification pipeline After obtaining the index set I, we retain the corresponding columns, Xred = X[:,I] , and train a linear classifier (logistic regression or linear SVM). At test time, raw features are computed, unselected dimensions are discarded, and the reduced vector is fed to the trained model. No retraining of the diffusion operator is required.

Fig. 1. (Left) Bad distribution of features: random placement yields no discernible spectral structure. (Right) Good multimodal distribution: compact, wellseparated clusters reflect semantic or statistical similarity.

The complete inference pipeline consists of five stages: 1) Compute object-level NBSE fingerprints Ψ (Eq. (16)). 2) Transpose X, build a QC-LDPC feature graph, and compute its smallest Bethe–Hessian eigenvector ϕmin . 3) Apply Algorithm 1 to obtain the index set I. 4) Train a linear classifier on the reduced matrix X[:,I] . 5) At test time, extract features, select columns I, and classify.

Both models were pre-trained on ImageNet and evaluated on the ImageNet-1K validation set (1000 classes, 50 000 images). To probe robustness under varying representation quality, we created two regimes: • Best representation: graph with 35 000 nodes; training classifier on 30 000 frozen embeddings. • Worst representation: graph with 25 000 nodes; training classifier on 20 000 frozen embeddings.

E. QC-LDPC graph construction

B. Graph construction

The quality of NBSE depends critically on graph topology. We construct a multi-edge type quasi-cyclic low-density parity-check (MET QC-LDPC) similarity graph [16] with the following properties: • Sparsity: Average degree davg ≈ 18, ensuring O(M ) edges and fast sparse eigenvalue solvers. • Quasi-cyclic structure: The adjacency matrix is a block circulant lift of a small protograph, which guarantees regular spectral properties and enables efficient construction via the method in [13]. • Large girth: Short cycles are avoided (girth ≥ 6), making the graph locally tree-like. This is essential because the Bethe approximation underlying Eq. (4) is asymptotically exact on trees. • Multi-edge type: Different edge types encode different similarity scales, improving the resolution of βN estimation. These constraints ensure that the graph ensemble has a welldefined degree distribution and concentrates its spectral properties. In practice, βN is stable across random lifts of the same protograph, contributing to the experimental robustness reported in Section V.

For every dataset we built one quasi-cyclic LDPC similarity graph: • Average degree davg ≈ 18, 2 • Edge weight Wij = exp(−∥xi −xj ∥ /(σi σj )) (Eq. (15)), • Local scale σi computed as the mean distance to the k = 10 nearest neighbours. The graph satisfies the numerical constraints of an (m, L) MET QC-LDPC code [16], guaranteeing fast eigenvalue computation using the method in [13].

V. E XPERIMENTAL E VALUATION A. Setup We used two deep-network encoders as feature generators: • MobileNetV2 [18]: 1280-dimensional penultimate layer. • EfficientNet-B4 [19]: 1792 dimensions.

C. Baselines We compared three feature-reduction strategies: 1) NBSE topology-informed diffusion-guided spectral ablation (Algorithm 1); 2) ANOVA F -test—select the top-n features with highest F scores [4]. Implemented via scikit-learn [17]; 3) Random choice—uniformly random selection of n features. For each method we varied the retained feature proportion p ∈ {1.0, 0.9, . . . , 0.3} and measured classification accuracy of a logistic-regression model (Crammer–Singer multi-class formulation). Each experiment was repeated five times with different random seeds; mean ± std is reported. D. Results We evaluated two variants for estimating βN : 1) Global quasi-stationary—compute a single βN on the full object graph and reuse it for all features (left panels in Figs. 2, 3);

Algorithm 1 Spectral Diffusion-Guided Ablation of Features Require: X ∈ RM ×D , target number n of retained features, QC-LDPC graph parameters. Ensure: Representative index set I, |I| = n. 1: // Stage 1: Feature-level spectral embedding 2: Z ← X⊤ ▷ Treat features as D objects in RM 3: Build feature affinity matrix C ∈ RD×D from Z via Eq. (15). 4: Construct D̃(β) and S(β) from C using
Eq. (4). 5: βN ← root of λmin I + D̃(β) − S(β) = 0 6: H ← I + D̃(βN ) − S(βN ) 7: ϕmin ← eigenvector for smallest eigenvalue of H ▷ ϕmin ∈ RD 8: if n ≤ 0 then return ∅ 9: else if n ≥ D then return {1, . . . , D} 10: // Stage 2: Balanced histogram binning 11: amin ← mini (ϕmin )i , amax ← maxi (ϕmin )i 12: if amin = amax then return first n indices 13: Partition [amin , amax ] into n equal intervals B0 , . . . , Bn−1 . 14: for k = 0 to n − 1 do 15: Ck ← {i : (ϕmin)i ∈ Bk } 16: qk ← n |Ck |/D ▷ Proportional quota, rounded P 17: Adjust quotas {qk } so that k qk = n (add/subtract remainder to/from largest bins). 18: I ← ∅, U ←∅ ▷ Used-feature tracker 19: for k = 0 to n − 1 do 20: if qk = 0 then continue 21: Ck′ ← Ck \ U 22: if Ck′ = ∅ then ▷ Fallback: nearest unused feature to bin centre mk 23: j ⋆ ← arg minj ∈U / |(ϕmin )j − mk | 24: else 25: Sort Ck′ by (ϕmin )i ascending. 26: Select qk indices uniformly from the sorted list (including end points). 27: Add selected index(indices) to I and mark as used. 28: return I

(l)

2) Independent per-feature—compute a separate βN for each univariate feature slice (right panels). The global variant simplifies computation and yields virtually identical classification performance, confirming that the Nishimori temperature is stable across feature slices under the quasi-stationarity hypothesis. This empirical observation is consistent with Proposition 2: if inter-feature perturbations are small relative to cluster separation, βN shifts negligibly. Figs. 2 (MobileNetV2) and 3 (EfficientNet-B4) show classification accuracy versus retained feature proportion. The main observations are: MobileNetV2. The spectral selection method on the best representation exhibits high stability under feature reduction, with accuracy fluctuations no greater than 5%; for the worst representation the fluctuation is below 3%. On the spectral

embedding, ANOVA and random selection perform more reliably than on original data. Random selection unexpectedly outperforms ANOVA, likely because ANOVA’s greedy nature selects overly correlated features. Working directly with original data leads to severe degradation: the model loses up to 17% accuracy with random selection and up to 25% with ANOVA. EfficientNet-B4. With this more powerful architecture, all methods gain stability. The best result is the combination of spectral embedding and NBSE selector on a good representation: accuracy loss is less than 1% even at 70% feature reduction (30% retention). This outperforms random selection on spectral embedding by 0.5%, ANOVA on spectral embedding by 2.7%, and both original-data methods by a significant 6.8%. These results confirm that the synergy of spectral data representation and specialised selection preserves information content during aggressive compression, especially with efficient neural architectures. VI. D ISCUSSION The experimental results provide compelling evidence that degree-corrected, diffusion-based feature selection offers distinct advantages over classical statistical filters for highdimensional deep-learning embeddings. Global correlation structure.: NBSE captures global correlations that univariate methods inherently miss. Classical filter methods such as the ANOVA F -test operate under an implicit independence assumption, scoring each dimension solely on its own variance relative to class labels. Deepnetwork embeddings are highly redundant; multiple neurons often encode semantically similar information because of over-parameterisation. By constructing a similarity graph and analysing the Bethe–Hessian spectrum, NBSE models the manifold structure of the data directly. The Nishimori identities guarantee that at βN thermal averages satisfy exact gauge symmetries, while the degree-corrected diagonal D̃ii (β) ensures that selected features are not merely those with high marginal variance but those contributing most significantly to the global diffusion process. Consequently, the intrinsic geometry of class clusters is preserved even when 70% of dimensions are discarded. Stability and representation quality.: A striking finding is the disparity in stability between spectral embedding space and original data space. For MobileNetV2, aggressive ANOVA reduction on raw features caused accuracy drops up to 25%, whereas NBSE maintained fluctuations within a narrow 5% band for good representations. The diffusion process acts as a regulariser, smoothing out high-frequency noise that typically destabilises greedy selection algorithms. Even with a suboptimal data slice, the spectral method exhibited only 3% fluctuation, indicating that the embedding itself provides a robust substrate less sensitive to sampling variance than raw feature vectors. The ANOVA paradox in spectral space.: Random selection outperformed ANOVA within the spectral embedding. This counter-intuitive result arises because diffusion embeddings transform features into a coordinate system where

Fig. 2. MobileNetV2: (left) quasi-stationary case, (right) non-quasi-stationary case—representation quality versus the percentage of preserved features. Solid blue line: NBSE spectral feature selector, best representation. Solid orange line: NBSE spectral feature selector, worst representation. Dashed red line: NBSE + ANOVA under Spectral Embedding (SE), best representation. Dashed gray line: NBSE + random selection, best representation. Dashed green line: original data + ANOVA. Dotted purple line: original data + random selection.

Fig. 3. EfficientNet-B4: (left) quasi-stationary case—single βN computed on the full object graph and reused for all features; (right) non-quasi-stationary (l) case—separate βN per feature slice. The near-identical curves validate the quasi-stationarity hypothesis, justifying the simplified global computation for production use. Line styles as in Fig. 2.

dimensions are highly correlated by design (reflecting shared diffusion modes). The greedy nature of ANOVA selects topscoring features independently, thereby picking multiple members of the same diffusion cluster and yielding diminishing returns. Random selection inadvertently samples across distinct diffusion modes. NBSE’s spectral ablation explicitly bins the eigenvector ϕmin and selects representatives from disjoint intervals, ensuring diversity. Architectural implications.: The performance gap between MobileNetV2 and EfficientNet-B4 underscores how feature selection interacts with backbone architecture. EfficientNet-B4’s superior stability (less than 1% loss at 70% reduction) suggests that deeper, more efficient networks produce denser, semantically richer embeddings better suited for spectral analysis. As backbone networks improve, the efficacy of physics-guided feature selection increases. Moreover, validation of the quasi-stationary hypothesis demonstrates computational viability: computing a single βN reduces complexity from O(D · M log M ) to O(M log M ), which is feasible for real-time applications where per-feature optimisation would be prohibitive.

Limitations and future directions.: NBSE relies on constructing a sparse similarity graph; in extremely high-noise regimes where manifold structure is obscured, Proposition 2 warns that βN may shift appreciably. Additionally, the current implementation focuses on binary spectral clustering via the smallest eigenvector; extending to multi-class Nishimori formulations could enhance performance on datasets with complex class boundaries. Future work will explore adaptive graph construction in which both protograph topology and edge weights are learned directly from data. In summary, NBSE is a principled approach grounded in statistical physics that aligns feature selection with the intrinsic degree-corrected diffusion properties of the data manifold. VII. C ONCLUSION We introduced Noise-Based Spectral Embedding (NBSE), a diffusion-guided feature-selection technique grounded in statistical physics. By exploiting the Nishimori temperature as an analytically determined optimal diffusion parameter and using the full Bethe–Hessian—including its cavity-susceptibility diP agonal D̃ii (β) = j sinh2 (βWij ) and effective off-diagonal

couplings Sij (β) = sinh(βWij ) cosh(βWij )—to capture the slowest heat-kernel mode, NBSE provides (a) a principled onedimensional spectral coordinate for each object; (b) a spectral ablation procedure that groups redundant features without greedy heuristics; (c) robust performance under realistic noise and aggressive dimensionality reduction. Proposition 2 rigorously bounds the shift in the Nishimori temperature caused by coloured Gaussian perturbations, confirming that the method remains stable for noise levels typical of deep embeddings. The corrected diffusion interpretation via Eq. (10) shows that the degree-corrected Bethe–Hessian avoids the spectral bias of standard Laplacians on sparse heterogeneous graphs—the regime where classical spectral methods fail. Experimental results confirm that the synergy of spectral data representation and specialised selection preserves information during significant compression. On EfficientNet-B4, accuracy loss was limited to less than 1% even at 70% feature reduction, outperforming baselines by up to 6.8%. Future work will scale NBSE to larger datasets (e.g. ImageNet-21K), develop adaptive graph-construction procedures that learn both protograph topology and edge weights from data, investigate ensembles of graph representations, and extend the framework to multi-class Nishimori formulations with non-binary label priors leveraging non-binary QC-LDPC codes. R EFERENCES [1] I. T. Jolliffe, Principal Component Analysis, 2nd ed. New York, NY, USA: Springer, 2002. [2] L. van der Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res., vol. 9, pp. 2579–2605, 2008. [3] L. McInnes and J. Healy, “UMAP: Uniform manifold approximation and projection for dimension reduction,” arXiv:1802.03426, 2018. [4] I. Guyon and A. Elisseeff, “An introduction to variable and feature selection,” J. Mach. Learn. Res., vol. 3, pp. 1157–1182, 2003. [5] R. A. Fisher, “The use of multiple measurements in taxonomic problems,” Annals Eugenics, vol. 7, no. 2, pp. 179–188, 1936. [6] R. Kohavi and G. John, “Wrappers for feature subset selection,” Artif. Intell., vol. 97, nos. 1–2, pp. 273–324, 1997. [7] H. Zou and T. Hastie, “Regularization and variable selection via the elastic net,” J. R. Stat. Soc. B, vol. 67, no. 2, pp. 301–320, 2005. [8] A. Y. Ng, M. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” in Advances in Neural Information Processing Systems, vol. 14, 2002, pp. 849–856. [9] M. Belkin and P. Niyogi, “Laplacian eigenmaps for dimensionality reduction and data representation,” Neural Comput., vol. 15, no. 6, pp. 1373–1396, 2003. [10] X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-supervised learning using Gaussian fields and harmonic functions,” in Proc. Int. Conf. Machine Learning, 2003, pp. 912–919. [11] L. Dall’Amico, R. Couillet, and N. Tremblay, “A unified framework for spectral clustering in sparse graphs,” J. Mach. Learn. Res., vol. 22, no. 217, pp. 1–56, 2021. [12] V. S. Usatyuk, D. A. Sapozhnikov, and S. I. Egorov, “Enhanced image clustering with random-bond Ising models using LDPC graph representations and Nishimori temperature,” Moscow Univ. Phys. Bull., vol. 79, suppl. 2, pp. S647–S665, 2024. [13] V. S. Usatyuk, D. A. Sapozhnikov, and S. I. Egorov, “Natural image classification via quasi-cyclic graph ensembles and random-bond Ising models at the Nishimori temperature,” Moscow Univ. Phys. Bull., vol. 80, suppl. 3, pp. S1039–S1053, 2025. [14] H. Nishimori, Statistical Physics of Spin Glasses and Information Processing: An Introduction. Oxford, U.K.: Oxford Univ. Press, 2001.

[15] A. Saade, F. Krzakala, and L. Zdeborová, “Spectral clustering of graphs with the Bethe Hessian,” in Advances in Neural Information Processing Systems, vol. 27, 2014, pp. 406–414. [16] T. J. Richardson and R. L. Urbanke, “Multi-edge type LDPC codes,” presented at the Workshop honoring Prof. Bob McEliece, Pasadena, CA, USA, 2002. [17] F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011. [18] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2018, pp. 4510–4520. [19] M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proc. Int. Conf. Machine Learning, 2019, pp. 6105–6114.