arXiv CSopen access

Normal Alignment: Improved Cryptanalytic Sign Recovery on Hard-Label Networks

· arxiv_cs

arXiv CS · Papers · License: Open Access

cryptography cybersecurity privacy security

cryptography, security, privacy, cybersecurity

Normal Alignment: Improved Cryptanalytic Sign Recovery on Hard-Label Networks Shi Tang1 , Zirui Chen2 , Yongjia Su1 , Zhengchao Gao1 , Lingyue Qin2 , and Xiaoyang Dong2 1

arXiv:2609.18751v1 [cs.CR] 16 Sep 2026

Shandong University, Jinan, P. R. China {shi.tang,yongjia.su,chao qwq}@mail.sdu.edu.cn 2 Tsinghua University, Beijing, P. R. China [email protected] {qinly,xiaoyangdong}@tsinghua.edu.cn

Abstract. At EUROCRYPT 2025, Carlini et al. proposed a breakthrough in the cryptanalytic extraction on hard-label (S1) deep neural networks (DNNs), demonstrating polynomial-time signature and sign recovery. However, Carlini et al.’s sign-recovery method (which we call Future Toggle) suffers only a marginal advantage over random guessing, producing high-confidence wrong sign predictions in deeper layers. Such errors trigger expensive exponential-time enumeration. This work presents Normal Alignment, a novel statistical sign-recovery approach for S1 DNNs. Drawing on the expected length difference between projected normals of adjacent decision facets at dual points, our method infers neuron signs via normal-signature alignment. It delivers higher voting accuracy and pushes erroneous predictions to low-confidence ranks, which further enables a more efficient combined method, eSOE + Alignment, by combining Normal Alignment with the hard-label SOE extension. This combined strategy removes heavy enumeration overhead and realizes exact polynomial-time full sign recovery. Experiments demonstrate the effectiveness of our method, especially for deep layers. For example, with our method, the signs for CIFAR-10 (architecture 192-64×8-10) and MNIST (architecture 64-96×3-32-10) models can be fully recovered in polynomial time; in contrast, Carlini et al.’s sign-recovery method would require exponential-time enumerations involving 252 or 282 guesses of the signs, respectively. Keywords: Cryptanalytic model extraction · ReLU networks · Sign recovery · S1 access · Normal Alignment

Introduction

Deep neural networks (DNNs) are widely used in computer vision [16], naturallanguage processing [24], and medical diagnosis [11], etc. Training high-performing DNNs often requires large amounts of data, computation and engineering efforts, making trained models valuable intellectual assets [20]. Model extraction is a long-studied attack in which an adversary uses input-output queries of the victim

S. Tang et al.

DNN (or other side-channel information [2]) to extract its parameters (weights and biases). Early works explored network reconstruction [12] and query-based model stealing [19,23]. At CRYPTO 2020, Carlini, Jagielski, and Mironov introduced a cryptanalytic approach to model extraction [6], which exploits the piecewise-affine structure of a ReLU network and recovers its parameters layer by layer from raw-output queries. Layer-wise extraction proceeds in two stages: First, the signature recovery identifies the unsigned weights and biases of each neuron by using the high-order differential at a so-called critical point, where one ReLU input of the DNN is exactly zero; Second, the sign recovery determines the sign. Their method enjoys a polynomial-time signature recovery phase, but suffers from an exponential-time sign recovery phase by a brute-force guessing method. At EUROCRYPT 2024, Canales-Martı́nez et al. [4] developed polynomial-time sign recovery algorithms in raw-output setting, i.e., the Neuron Wiggle and SOE methods. Later work examined the practical limitations of these layer-wise attacks. Foerster et al. [13] found that increasing the number of critical points does not necessarily improve Neuron Wiggle recovery for difficult neurons. Liu et al. [18] addressed rank-deficient signature systems and the misattribution of critical points from deeper layers, thereby extending practical extraction from three hidden layers to eight. In parallel, cryptanalytic extraction has expanded along two dimensions: covering various activation functions [8,21,1] and different network architectures [25,22,17,9]. Hard-label extraction. The S1 hard-label setting returns only the predicted class labels (e.g., “dog” or “car”) and hides the logits. At ASIACRYPT 2024, Yi Chen et al. initiated the cryptanalytic extraction in the S1 setting [7], but it requires exponential execution time. At EUROCRYPT 2025, Carlini et al. [5] gave the first polynomial-query, polynomial-time hard-label extraction. The attack collects and clusters dual points, which are critical and also on a visible class-decision boundary, to recover the signatures. At CRYPTO 2026, Ito, Miura, and Todo [15] identified a limitation of Carlini et al.’s attack [5]: for nearly always-active neurons, the state switches needed for parameter recovery can become exponentially difficult to observe. They proposed cross-layer extraction to address this failure mode. In 2026, Zirui Chen et al. [10] proposed the Approximate Signature Vector (ASV) method to reduce the cost of clustering dual points, and hence improve Carlini et al.’s signature recovery phase [5]. Existing Sign Recovery Methods in S1 and Their Limitations. There are only two existing S1 sign recovery methods: Carlini et al.’s statistical Future Toggle method [5], and Canales-Martı́nez and Santos’s deterministic hard-label SOE method [3]. The hard-label SOE typically recovers only the first hidden layer, unless the network is sufficiently contractive to permit the recovery of deeper layers. The main limitation of Future Toggle is its weak advantage over random guessing. In our practical experiments, Future toggle suffers small advantage than random guessing, leading to the low vote accuracy and the low vote confidence. For example, on the evaluated CIFAR-10 model, Table 2 re-

Normal Alignment for Sign Recovery on Hard-Label Networks 100

100

80 70

Confidence: 55.58% Confidence rank: 20/64

60 50

90 Confidence (%)

Confidence (%)

80 70 Confidence: 55.50% Confidence rank: 60/64

16 Correct

32 40 Neuron index

Wrong

Median

Tie

(a) Future Toggle (nattempt = 1000).

16 Correct

32 40 Neuron index

Wrong

Median

Tie

(b) Normal Alignment (nattempt = 200).

Fig. 1: vote confidence distributions of Future Toggle and Normal Alignment on the 7th hidden layer of the CIFAR-10 model with architecture 192-64×8-10.

ports vote accuracies of 52%–57%, while Figure 1a shows that the confidence of most neurons is below 60%. The weak advantage may lead to many incorrectly recovered signs in deep layers, which results in an exponential time complexity to recover the full signs, shifting the entire hard-label extraction from polynomial to exponential time complexity. Specifically, the Future Toggle may produce errors with relatively high confidence. As shown in Figure 1a, 9 of the 64 signs on layer 7 are incorrectly recovered, including one with a confidence of 55.58% that ranks 20th among all neurons by confidence. This makes exact recovery of the entire layer difficult under two existing approaches. – Enumeration Infeasible. To recover all signs, we follow the approach of Foerster et al. [13]: we enumerate and guess the low-confidence sign assignments (which may contain errors) until all erroneous signs are covered, and verify the assignments by executing the next-layer signature recovery algorithm.As shown in Figure 1a, since the erroneous sign ranks 20-th in confidence, the enumeration must cover all 45 signs from rank 20 to rank 64. This requires testing 245 possible assignments. – SOE Failure. In 2026, Liu et al. [18] combined statistical predictions (Neuron Wiggle) with SOE in raw-output setting by eliminating unknowns associated with neurons predicted to be inactive with high confidence. Hence, when an actually active neuron is incorrectly predicted to be inactive with high confidence, its nonzero unknown is likely to be eliminated, making the reduced SOE incorrect and causing the combined strategy to fail.

Our Contributions. This paper introduces a new sign recovery method in S1 setting, called Normal Alignment, which uses the normals of the two decision facets adjacent to a dual point to directly determine the target sign, without repeatedly walking along the decision boundary to search for neuron toggles in

S. Tang et al.

future layers (i.e., Carlini et al.’s Future Toggle method [5]). Our method is based on the following statistical intuition (which is also formally proved): at a dual point, the decision-facet normal on the active side has a larger expected length than the normal on the inactive side, because it explicitly includes the target neuron’s weight contribution. The advantages of our methods are summarized below: 1. Larger Statistical Advantage. As shown in Table 7, with the same budget of nattempt = 200 dual points, Normal Alignment achieves an overall vote accuracy of 68.74% across all hidden layers, compared with 60.89% for Future Toggle. In deep hidden layers in Table 6, our method still maintains a vote accuracy of about 70%, while the vote accuracy of Future Toggle drops to around 53% in layers 3-7. 2. Fewer Queries Needed. Normal Alignment neither walks along the decision boundary, nor discards dual points due to non-future neuron toggles – as occurs with Future Toggle. When the two adjacent decision-facet normals are successfully recovered, a dual point can produce a vote. Consequently, the dual point utilization reaches 100%, as shown in Table 6. By contrast, only 2.26% of the dual points produce a vote in Layer 8 by the Future Toggle. Furthermore, the weak advantage of the Future Toggle naturally requires more votes to improve accuracy, and hence usually needs more queries and time than Normal Alignment as shown in Table 1. 3. Low Confidence for Incorrect Signs. As shown in Figure 1, For Normal Alignment, the highest-confidence error has a confidence of 55.50% and ranks 60th among the 64 neurons, whereas Future Toggle’s highest-confidence error has a similar confidence of 55.58% but ranks as high as 20th. Therefore, to recover the full signs by enumeration [13], our method must test 25 sign assignments, while Future Toggle tests 245 sign assignments. We also test more models with different layers in Table 5 in Supp. A . Across all hidden layers of each model, our highest-ranked errors occur on CIFAR-10 L6 and MNIST L2 at ranks 50/64 and 68/96, respectively, yielding enumeration complexities of 215 and 229 . By contrast, at the same budget of nattempt = 200, the highest-ranked errors of Future Toggle require enumeration complexities of 264 and 293 on the CIFAR-10 and MNIST models, respectively. 4. Feasible Combination of the Hard-label SOE and Normal Alignment: eSOE + Alignment. Because the highest-confidence error ranks very low in Normal Alignment, a similar combination of a statistical method and a deterministic method by Liu et al. [18] in S5 setting works in S1 setting, i.e., combining hard-label SOE and Normal Alignment. The Normal Alignment identifies the high-confidence (highly ranked) inactive neurons and eliminates the corresponding zero equations in SOE. By contrast, Future Toggle is susceptible to high-confidence errors (i.e., errors with high confidence rank); for instance, an active neuron might be predicted as inactive with high confidence, causing a nonzero equation to be erroneously discarded and thus causing the SOE method to fail. Similarly to Liu et al. [18], to further increase the rank of SOE, we select several transition points

Normal Alignment for Sign Recovery on Hard-Label Networks

sharing the same activation states in the target and future layers, forming a stacked coefficient matrix. Besides, we also introduce an orthogonal projection matrix to eliminate the unknown normal length at each point. Therefore, we call the resulting method hard-label SOE extension and Normal Alignment (eSOE + Alignment). It helps eliminate the enumeration complexities – specifically the 215 time for layer L6 of the CIFAR-10 network (Table 6) and the 229 time for layer L2 of the MNIST network (Table 7). Experiments. As shown in Table 1, the sign recovery methods are evaluated on CIFAR-10 and MNIST networks. We follow the same assumption as [4]: when targeting the layer k, the preceding layers (< k) and the unsigned signatures of layer k are known. Tables 6 and 7 in Supp. A report the full results of our experiments, comparing Normal Alignment, eSOE + Alignment, Future Toggle, the combination of the hard-label SOE extension and Future Toggle (eSOE + Toggle) for a fair comparison though eSOE + Toggle does not reduce the overall enumeration complexities as shown in Table 1. Our eSOE+Alignment correctly recovers all 512/512 signs on CIFAR-10 and all 320/320 signs on MNIST, thereby achieving exact sign recovery in polynomial time in Table 1. In contrast, eSOE+Toggle recovers only 469/512 and 296/320 signs, respectively. Specifically, for the CIFAR-10 model with nattempt = 1000 in Table 6, eSOE + Toggle leaves errors on L5, L7, and L8, whose highest-confidence erroneous signs rank 44-th, 20-th, and 13-th out of 64, respectively. Exact recovery must therefore enumerate the 52 signs from rank 13 to rank 64 in L8, requiring 252 sign enumerations. For the MNIST model in Table 7, the highest-confidence error ranks 15-th among 96 neurons on L3, hence requiring 282 sign enumerations to recover full signs. The source code for all the experiments can be found via XXX

Preliminaries

Unless otherwise specified, the subscript and superscript numbers start from 1. – [m]: for a positive integer m, we write [m] = {1, . . . , m}, – A: matrix, where its i-th row is Ai , and its element in i-th row and j-th column is Ai,j , i, j ≥ 1, – x: column vector, and its i-th element is xi , i ≥ 1, – F: functions, – C, D: space or set, – neuron (k, j): the j-th neuron in layer k. 2.1

Notations and Definitions

The DNN is composed of a sequence of functions alternating between linear func(k) (k+1) tions f (k) : Rd 7→ Rd (k ∈ [r +1]), and a nonlinear function σ (componentwise ReLU function): Fθ = f (r+1) ◦ σ ◦ f (r) ◦ σ ◦ · · · f (2) ◦ σ ◦ f (1) ,

(1)

S. Tang et al.

Table 1: Comparison of our eSOE+Alignment and eSOE+Toggle [5] on the CIFAR-10 and MNIST models. Model and Method Architecture

Recovery Complexity nattempt

Method

Method execution

Complete recovery

Experimental Results Correct signs

Time 10.73

231.13

213.47 (s) + 252 (g)

235.49

211.20

228.71

CIFAR-10 192-64×8-10

eSOE+Alignment

200

poly

512/512

eSOE+Toggle [5]

1000

poly

exp

469/512

MNIST 64-96×3-32-10

eSOE+Alignment

200

poly

320/320

eSOE+Toggle [5]

1000

poly

exp

Queries

17.39

296/320

(s) + 2

(g)

230.77

nattempt : denotes the number of attempted dual points per neuron. As shown in Fig. 3, we set nattempt = 200 for Normal Alignment. Because Future Toggle provides a weaker statistical advantage, we set nattempt = 1000 for Future Toggle. Method execution: denotes the time complexity of eSOE+Alignment or eSOE+Toggle. Since hard-label SOE, Normal Alignment, and Future Toggle all run in polynomial time, both combined methods also run in polynomial time. Complete recovery: denotes the time complexity required to recover all neuron signs correctly. eSOE+Alignment recovers all neuron signs correctly and therefore achieves complete recovery in polynomial time. In contrast, eSOE+Toggle leaves some signs incorrect; guaranteeing complete recovery therefore requires exponential enumeration of the unresolved sign assignments [13]. Time: consists of two components. The notation 2x (s) denotes the time of eSOE+Alignment/Toggle in seconds, whereas 2y (g) denotes the cost of sign guessing required for complete recovery of all signs in the model. Queries: In the CIFAR-10 proof-of-concept experiments, the decision-facet normals are computed directly from model parameters, and the reported values estimate the corresponding hard-label query cost; the MNIST entries report the actual hard-label query counts.

(k)

where f (k) : Rd

(k+1)

→ Rd

is an affine transformation: (k+1)

y (k) = f (k) (x(k) ) = A(k) x(k) + b(k) ∈ Rd

(2)

(k)

(1)

where x(k) ∈ Rd represents the input vector of layer k, and x(1) ∈ Rd is (k+1) ×d(k) the model input. The weight matrix A(k) ∈ Rd and the bias vector (k+1) b(k) ∈ Rd are composed of floating-point numbers, which are the model parameters. (1) Given input x(1) ∈ Rd , the ReLU function σ in layer k is also interpreted as the matrix determined by x(1) , (k)

(k)

I (k) = diag(τ1 , τ2 , . . . , τd(k+1) ), (k)

(k)

where τi = 1, i ∈ [d(k+1) ] when yi ≥ 0, else τi (k+1) I (k) y (k) ∈ Rd is the output of layer k.

(3) = 0. Then, x(k+1) =

(1)

Definition 1 (Linear Neighborhood). Given an input x ∈ Rd , the matrices A(k) , b(k) , I (k) , k ∈ [r + 1] will be all fixed. The linear neighborhood of x is (1) defined as the subset Lx ⊂ Rd , so that, for all x(1) ∈ Lx , the same matrices A(k) , b(k) , I (k) will be applied to compute the output of the DNN. The DNN has been proved to be a piecewise linear function [6,4], i.e., for x ∈ Lx , the model output will change linearly, i.e., the DNN is reduced to Fθ (x) = A(r+1) I (r) A(r) · · · I (1) A(1) x + b(1) · · · + b(r) + b(r+1) = A(r+1) I (r) A(r) · · · I (2) A(2) I (1) A(1) x + β = Γ x + β.

(4)

Normal Alignment for Sign Recovery on Hard-Label Networks

Definition 2 (Oracle models). In S5 raw-output setting, the oracle returns all logits: OS5 (x) = Fθ (x). In S1 hard-label setting, it returns only the label OS1 (x) = min arg max Fθ (x), j∈[d(r+2) ]

where the minimum indicates a deterministic rule for ties. Definition 3 (Critical hyperplane, activation boundary, and critical point). The critical hyperplane of neuron (k, j) (the j-th neuron of layer k) in the input space of layer k is (k) (k) (k) (k) Hj = u ∈ Rd : ⟨Aj , u⟩ + bj = 0 , k ∈ [r + 1], j ∈ [d(k+1) ] . The corresponding activation boundary in the model-input space is (1) (k) (k) Cj = x(1) ∈ Rd : x(k) ∈ Hj . (k)

Therefore, x(1) ∈ Cj

is a critical point of the neuron (k, j).

Prefix and suffix maps of layer k. Fix a target layer k ∈ [r + 1]. We decompose the network as Fθ = f (r+1) ◦ σ ◦ f (r) ◦ · · · ◦ σ ◦ f (k+1) ◦σ ◦ f (k) ◦ σ ◦ f (k−1) ◦ · · · ◦ σ ◦ f (1) , (5) | | {z } {z } k+1 Gx

Fxk−1

where Fxk−1 is the layers before layer k, and Gxk+1 is the layers after the layer k. Given model input x and its corresponding linear neighborhood Lx , the Fxk−1 and Gxk+1 collapse to fix affine functions, i.e., for model input x(1) ∈ Lx , and its corresponding x(k) , x(k) = Fxk−1 (x(1) ) = F (k−1) x(1) +β (k−1) , Gxk+1 (x(k+1) ) = G(k+1) x(k+1) +γ (k+1) . (6) where F (k−1) = I (k−1) A(k−1) · · · I (1) A(1) , G(k+1) = A(r+1) I (r) A(r) · · · I (k+1) A(k+1) . according to Eq. (4), and hence Eq. (4) becomes Fθ (x(1) ) = G(k+1) I (k) A(k) F (k−1) x(1) + β (k−1) + b(k) + γ (k+1) . (7) Definition 4 (Transition and dual points, decision boundary, decision facet). For distinct classes a, b ∈ [d(r+2) ], define Dab (x) = Fθ (x)a − Fθ (x)b , and the set of decision boundary (1) Dab = x ∈ Rd : Fθ (x)a = Fθ (x)b > Fθ (x)c for every c ∈ [d(r+2) ]/{a, b} . (8) Any x ∈ Dab is a transition point for switching classes a and b, whose decision (1) (1) facet in the input model space Rd is defined as Dab ∩ Lx ∈ Rd . The point (k) x ∈ Dab ∩ Cj is a dual point for neuron (k, j) and class pair (a, b). Usually, there are at least two adjacent decision facets for a given dual point x, denoted as Dab ∩ Lx and Dab ∩ L′x .

S. Tang et al.

Figure 2 summarizes different types of point in geometry, where each cell represents a linear neighborhood.

Decision boundary Dab Activation boundaries Linear Neighborhood critical point

L′x

transition point dual point

Fig. 2: Piecewise-affine geometry in the model-input space.

Definition 5 (Layer-k wiggle). A layer-k wiggle around a model input x is a (k) (1) small vector δ (k) ∈ Rd , so that there exists δ (1) ∈ Rd that satisfies x + δ (1) ∈ Lx (or x − δ (1) ∈ Lx ) and δ (k) = F (k−1) (x + δ (1) ) + β (k−1) − (F (k−1) (x) + β (k−1) ) = F (k−1) (δ (1) ) (or δ (k) = F (k−1) (x) + β (k−1) − (F (k−1) (x − δ (1) ) + β (k−1) ) = F (k−1) (δ (1) )). Definition 6 (Control space). Given a model input x and its linear neighborhood Lx , its control space at the input to layer k is n o (1) (k) (k) V(k) = F (k−1) δ (1) : δ (1) ∈ Rd , x + δ (1) ∈ Lx ⊆ Rd . x := span δ (k)

Equivalently, Vx is the subspace spanned by the columns of F (k−1) . Definition 7 (Projection onto the control space). Since the control space ⊥ (k) (k) = Vx is the column space of F (k−1) , its orthogonal complement satisfies Vx (k) ⊤ ker F (k−1) . Every vector u ∈ Rd can therefore be uniquely written as (k)

(k)

u = u + u⊥ , where u ∈ Vx and u⊥ ∈ (Vx )⊥ . We call u the orthogonal (k) projection of u onto Vx and write u := P (k) u, where P (k) is the corresponding orthogonal projection matrix. Equivalently, if the columns of Q(k) form an (k) orthonormal basis of Vx , then P (k) = Q(k) Q(k)

⊤

u = Q(k) Q(k)

In particular, P (k) is symmetric, i.e., (P (k) )⊤ = P (k) .

⊤

Normal Alignment for Sign Recovery on Hard-Label Networks

2.2

Extraction Goal and Assumptions (k)

Definition 8 (Signature [4]). Let Aj

(k)

= (Aj,1 , . . . , Aj,d(k) ) be the weight (k)

vector of neuron (k, j), and assume that Aj,1 ̸= 0. Its signature is the vector 



(k) (k) (k) Aj,d(k) Aj,2 Aj (k) b  , Aj := (k) = 1, (k) , . . . , (k) Aj,1 Aj,1 Aj,1 (k)

where the true weight vector satisfies Aj

(k)

(9)

(k)

b . = Aj,1 A j

Thus, after signature recovery, the only remaining ambiguity is the nonzero (k) scalar Aj,1 . Its magnitude does not need to be recovered [6]: since ReLU(cz) = c ReLU(z) for every c > 0, a positive scaling can be absorbed into the outgoing weights of the neuron. Its sign, however, is essential. Negating the recovered affine form exchanges its active and inactive sides and cannot be absorbed through (k) ReLU. We call the sign of Aj,1 the sign of neuron (k, j). Sign recovery determines this sign and thereby identifies the true active side of the neuron. Denote signs in the layer k by (s1 , s2 , · · · , sd(k+1) ), sj ∈ {1, −1}, ∀j ∈ [d(k+1) ], and define the sign matrix (k) (k) (k) S (k) = diag(s1 , s2 , . . . , sd(k+1) ). (10) (1)

Then, given x ∈ Rd , by Eq. (7), the model output in S5 setting is b(k) F (k−1) x + β (k−1) + b b(k) + γ (k+1) , Fθ (x) = G(k+1) I (k) S (k) A

(11)

b(k) and b where A b(k) are the unsigned signatures and biases in layer k. Extraction Goal. Our final objective is functionally equivalent parameter extraction: given oracle access to a target network Fθ , recover parameters θ̂ such that the extracted network computes the same function as the target, up to unavoidable symmetries such as positive neuron rescaling and permutation within a layer. This paper focuses on sign recovery in S1 hard-label setting and assumes that a preceding signature-recovery phase has recovered the target signatures up to nonzero scalar multiples. Assumptions. – Known architecture. The attacker knows (d(1) , . . . , d(r+2) ) and that the hidden layers are fully connected ReLU layers. (1) – Full-domain inputs. The attacker may adaptively query any input in Rd . – Precise computation. The analysis assumes exact real arithmetic or sufficiently high floating-point precision. – Oracle access. We consider both S5 raw-output and S1 hard-label access.

S. Tang et al.

– Available signatures. For the target layer k, we assume that all the weights and biases of the preceding layers 1, · · · , k − 1 are recovered, while each b(k) and bias b neuron’s signature A b(k) are known up to an unknown nonzero scalar in layer k. Also, we assume that no two signatures are the same [4]. Our goal is to recover the signs in layer k.

Existing Sign Recovery Methods in S1 Setting and their Limitations

3.1

Future Toggle in Hard-label Setting

Neuron Wiggle in S5 Setting [4]. The Neuron Wiggle method, proposed by Canales-Martı́nez et al. at EUROCRYPT 2024, is a heuristic sign recovery method in raw-output setting. The method relies on a basic asymmetry across the target activation boundary: the norm of the target layer output change is larger on the target active side because ReLU blocks the target neuron’s contribution on the inactive side. We first establish this asymmetry and then explain how it motivates the observation used by Future Toggle [5] in S1 setting. Suppose that x is a critical point of the target neuron (k, j), and denote its two adjacent linear neighborhoods by Lx and L′x . Their activation statuses differ only in the state of neuron (k, j). Without loss of generality, suppose that the target neuron is active in Lx and inactive in L′x . Denote the corresponding (1) (k) (k) activation matrices in layer k by I+ and I− . Choose a wiggle δ (1) ∈ Rd such that x + δ (1) ∈ Lx and x − δ (1) ∈ L′x . It induces the layer-k wiggle δ (k) := (k) F (k−1) δ (1) by Def. 5. Hence, the layer-(k + 1) wiggles are δ (k+1) = I+ A(k) δ (k) (k) and δ ′(k+1) = I− A(k) δ (k) on the active and inactive sides, respectively. Let S denote the set of active neurons in layer k within Lx . We have X (k) X (k) ∥δ (k+1) ∥2 = |Ai δ (k) |2 , ∥δ ′(k+1) ∥2 = |Ai δ (k) |2 . (12) i∈S

i∈S\{j}

Consequently, (k)

∥δ (k+1) ∥2 = ∥δ ′(k+1) ∥2 + |Aj δ (k) |2 .

(13)

(k) Therefore, whenever Aj δ (k) ̸= 0, the layer-(k + 1) wiggle has a strictly larger

norm on the active side. For one coordinate of the model output vector (e.g., the first coordinate), Eq. (7) gives the following output changes: (k+1)

(r+2)

δ1 := (Fθ (x + δ (1) ) − Fθ (x))1 = G1 δ (k+1) , ′(r+2) (k+1) ′(k+1) (1) δ . δ1 := (Fθ (x) − Fθ (x − δ ))1 = G1 (r+2)

′(r+2)

(k+1)

(14)

(k)

(k)

when δ (k) is parallel to Aj . Since δ (k) ∈ Vx according to Def. 6 and Aj

̸∈

Normal Alignment for Sign Recovery on Hard-Label Networks

11 (1)

(k)

Vx , the Neuron Wiggle thereby chooses the direction of the wiggle δ (1) ∈ Rd , (k) to have a layer-k wiggle δ (k) , which is exactly parallel to the projection of Aj (k)

to Vx . This wiggle strengthens the target neuron (k, j)’s contribution, making the absolute network output change on the active side more likely to be larger. It therefore predicts the side with the larger absolute output change to be the target active side [4]. Since a single comparison depends on several factors, notably the network architecture and the neuron activation states around x, an individual vote is not guaranteed to be correct. Neuron Wiggle thereby aggregates votes from many critical points to improve the accuracy of sign recovery. Motivation from Neuron Wiggle: Future Toggle [5]. In the hard-label setting, the attacker cannot observe changes in the output logits. At EUROCRYPT 2025, Carlini et al. [5] instead compare the walking distance from the two sides of a dual point to the first future-layer neuron toggle. When a boundary walk crosses a neuron’s activation boundary, its activation state changes and the visible decision boundary bends, as illustrated in Fig. 2. We call such an activation-state change a neuron toggle. The side that reaches a future-layer toggle after a shorter distance is predicted as the target active side. The distance comparison is motivated by the statistical signal exploited by Neuron Wiggle. Conceptually, consider two opposite input perturbations δ (1) and δ ′(1) = −δ (1) from the dual point x. The corresponding layer-(k + 1) wiggles δ (k+1) and δ ′(k+1) satisfy Eq. (12). For a future neuron (p, q) with p > k, the corresponding changes in its preactivation on the two sides are (p) (p) Aq I (p−1) · · · A(k+1) δ (k+1) and Aq I (p−1) · · · A(k+1) δ ′(k+1) respectively. For the same distance ∥δ (1) ∥, the ∥δ (k+1) ∥ on the active side is larger than ∥δ ′(k+1) ∥ on the inactive side according to Eq. (13), hence a future neuron’s preactivation tends to change faster on the active side. Although hard-label access doesn’t reveal preactivation’s rate of change, it reveals the resulting activation-state change when the preactivation crosses zero. Future Toggle therefore uses the distance to this toggle as an indirect proxy for the unobservable rate of change. (k) To strengthen this effect, the input perturbation should maximize Aj F (k−1) δ (1) . (k)

The ideal input-space direction is therefore parallel to (Aj F (k−1) )⊤ . However, the attacker must remain on the visible decision boundary in order to detect its bends. For an adjacent decision facet with unitD normal n(1) , the walking E (k) (k) direction is thus chosen parallel to (Aj F (k−1) )⊤ − (Aj F (k−1) )⊤ , n(1) n(1) . Starting from the dual point x, the attacker follows the projected direction on one adjacent decision facet until the decision boundary bends. Using the recovered parameters of layers 1, . . . , k, the attacker checks whether the bend is caused by a neuron in one of these layers. If so, it adds the current segment length(i.e., the Euclidean distance from the previous bend, or from x for the first segment) to the accumulated distance, relocates onto the adjacent decision facet, recomputes the projected direction, and continues walking. Otherwise, the bend is attributed to a future-layer neuron toggle and the walk terminates. The

S. Tang et al.

same procedure is applied on the other side of x, and the side with the shorter accumulated distance is predicted as the target active side. Limitations of the Future Toggle. Future Toggle [5] attempts to identify the target active side by comparing the distances from a dual point to the first future layer toggles on its two sides. However, obtaining these distances and using them for sign recovery introduce limitations in both efficiency and accuracy. To facilitate our discussion, we perform a sign recovery experiment using Future Toggle method [5] on the CIFAR-10 network with architecture 3072-256×3-64-10. The white-box information is used to identify bends caused by neurons in the recovered layers and to evaluate whether each valid vote is correct. Table 2 summarizes the resulting boundary walking and voting statistics. The column of “Bends from layers 1, · · · , k” reports the number of bends (neuron toggles) from recovered layers encountered by walking from each attempted dual point. Entries in Table 2 are reported as “the mean ± standard deviation” values across the ten selected neurons.

Table 2: White-box diagnostic of Future Toggle on the CIFAR-10 network 3072-256×3-64-10. For each hidden layer, ten neurons are randomly selected, with 1000 attempted dual points evaluated for each of these neurons. Layer

Bends from layers 1, . . . , k

Dual point utilization (%) ηdual = nvalid /nattempt

Vote accuracy (%) p = ncorrect /nvalid

1 2 3 4

0.87 ± 0.22 1.58 ± 0.08 6.73 ± 0.25 12.45 ± 0.52

99.74 ± 0.18 94.65 ± 1.63 68.18 ± 2.63 3.21 ± 0.53

54.14 ± 7.16 56.51 ± 3.15 55.31 ± 2.31 52.82 ± 8.08

– Limitation 1: Cost of decision boundary tracing. During each of the two walks in Dab ∩ Lx and Dab ∩ L′x from a dual point x, the attacker must detect when the current decision facet ends and the decision boundary bends. This information is not directly provided by the hard-label oracle. Instead, the attacker must repeatedly query the oracle to determine whether the walk remains on the same decision facet and use binary search to locate the bend when the facet changes. This cost is further amplified when a detected bend is caused by the neuron in a known layer 1, . . . , k rather than by a future layer. Such a bend does not terminate the walk. Instead, the attacker must recover the normal of the new decision facet and continue the walk. Each such bend requires the recovery of a new decision facet normal, which involves d(1) − 1 coordinate ratio searches, according to Eq. (22) in Sect. 4.1. Consequently, this cost can be substantial when the model input dimension is large (e.g., d(1) = 3072).

Normal Alignment for Sign Recovery on Hard-Label Networks

In the 2nd column of Table 2, when the layer depth increases, the mean number of bends from the recovered layers rises from 0.87 to 12.45, indicating that walks in deeper layers require more decision-facet normal recoveries. – Limitation 2: Limited utilization of dual points. A search may repeatedly encounter activation boundaries belonging to recovered layers or fail to reach a future layer toggle within the maximum searching distance. In either case, the dual point is discarded. Let nattempt denote the number of attempted dual points and nvalid the number that produce valid votes. The dual point utilization rate is defined as ηdual := nvalid /nattempt . A lower ηdual requires more attempted dual points and results in a longer time. In the third column of Table 2, as the layer depth increases, the dual point utilization rate falls from 99.74% to 3.21%. – Limitation 3: Weak statistical advantage. Future Toggle infers the target active side through two successive proxy relations: a larger target layer output change is expected to produce faster changes in future neurons, and the faster neuron value changes are expected to produce a shorter distance to the first future layer toggle. Neither relation is guaranteed to hold at every dual point, and the probability that the target active side produces the shorter distance may be only slightly greater than one half. Let p = 1/2+γ denote the probability that an individual valid vote is correct, where γ > 0 represents the statistical advantage in favor of the target active side. Assuming that the valid votes are independent and share the same success probability p, Hoeffding’s inequality gives Pr[the majority vote is incorrect] ≤ exp(−2γ 2 nvalid ). Therefore, ensuring an error probability of at most α requires nvalid ≥ ln(1/α)/(2γ 2 ), which grows with 1/γ 2 . For example, when p = 0.55, about nvalid = 103 valid votes are required to reach a confidence level of 99% [5]. This bound shows that a weaker statistical advantage requires more valid votes to reach the same confidence level. With a limited budget nattempt , more neurons may therefore remain below the required confidence threshold. In the 4th column of Table 2, ncorrect denotes the number of valid dual points that produce correct votes; the vote accuracy is p = ncorrect /nvalid . The average vote accuracy of each layer ranges from 52.82% to 56.51%. It shows that an individual valid vote is only slightly more likely to be correct than random guessing. – Limitation 4: Amplified limitations in the last hidden layer. The Future Toggle requires a different terminal event for the last hidden layer, where there is no future-layer ReLU neuron. It therefore continues each search until an intersection of decision boundaries and uses this distance instead [5]. This special treatment amplifies the preceding three limitations. First, a class decision boundary intersection may be farther from the dual point than a future layer neuron toggle, resulting in a longer walk. Second, the required intersection may not be reached within the maximum walking distance and the corresponding dual point is then discarded, reducing ηdual and increasing the number nattempt of attempted dual points. Finally, the two walks may terminate at intersections with different class decision boundaries, so their

S. Tang et al.

measured distances depend on different class decision boundaries as well as on the rates of logit change. In the last hidden layer of Table 2, only about 32 of the 1000 attempted dual points per neuron produce valid votes on average. 3.2

System of Equations (SOE) and Its Extensions

The Raw-Output SOE Method [4]. At EUROCRYPT 2024, Canales-Martı́nez et al. proposed the System of Equations (SOE) method in S5 setting. Specifically, targeting a single coordinate of the output vector (e.g., the first coordinate) at (1) (1) (1) an input x, the attacker samples d(k+1) perturbations δ1 , . . . , δd(k+1) ∈ Rd (1)

such that every perturbed input x + δℓ (ℓ ∈ [d(k+1) ]) remains within the same linear neighborhood as x. Then the linear system is built by computing the output difference between Fθ (x + δℓ ) and Fθ (x), ⊤    (1) b(k) F (k−1) δ (1) A (Fθ (x + δ1 ) − Fθ (x))1 1   ⊤     .. .. ,  G(k+1) I (k) S (k)  = 1   .   .  ⊤  (1) (k) (k−1) (1) (F (x + δ ) − F (x)) 1 b θ θ (k+1) A F δ (k+1) d 

(15)

b(k) , b where F (k−1) , A b(k) are known and the target is to recover S (k) . Solving (k+1) (k) (k) ⊤ this linear system yields the unknown vector (G1 I S ) . Since the ReLU activation suppresses negative values, any neuron t in the layer k that is inactive (k+1) (k) (k) ⊤ at x will have a corresponding entry of zero in (G1 I S ) . Once the inac(k) (k) b x(k) + b tive neurons are identified (e.g., neuron t), if A bt > 0, its sign will be t (k) st = −1. To ensure the linear system in Eq. (15) has a unique solution, the cob(k) F (k−1) ) = d(k+1) . efficient matrix must be of full rank, which requires rank(A (1) (k) (k+1) This implies d , . . . , d ≥ d . Consequently, this deterministic method is primarily applicable to network architectures that are sufficiently contractive. The SOE + Wiggle in Raw-Output Setting [18]. At EUROCRYPT 2026, Liu et al. combined the SOE and the Neuron Wiggle methods. As summarized in Sect. 3.1, Neuron Wiggle [4] recovers the sign of each neuron along with a confidence level, where a high confidence level strongly indicates a correct recovery. (k+1)

⊤

Recall that the unknown vector in Eq. (15) is expressed as (G1 I (k) S (k) ) = (k) (k+1) (k) (k+1) (k) (k) (G1,1 τ1 s1 , . . . , G1,d(k+1) τd(k+1) sd(k+1) )⊤ . If neuron t is identified as inactive (k) b(k) (k) (k) (k) (determined by evaluating st (A +b bt ) < 0, provided that the sign st t x is recovered correctly with a high confidence level by Neuron Wiggle), the cor(k+1) (k) (k) responding term G1,t τt st can be directly set to 0, thereby reducing the number of unknowns. Let K contain the neurons that Neuron Wiggle identifies as inactive at x with high confidence, and U := [d(k+1) ] \ K contain the remaining neurons. For a vector v, let [v]K and [v]U denote the subvectors indexed by

Normal Alignment for Sign Recovery on Hard-Label Networks

K and U, respectively. Since every neuron in K is inactive, the corresponding (k+1) (k) (k) ⊤ entries of (G1 I S ) are zero. The SOE thereby reduces to i⊤    b(k) F (k−1) δ (1) (1) A 1 (Fθ (x + δ1 ) − Fθ (x))1  U  ⊤     .. ..   G(k+1) I (k) S (k) . = 1   .   . h i⊤  U (1) (k) (k−1) (1) (F (x + δ ) − F (x)) 1 b θ θ A F δd(k+1) d(k+1)  h

(16)

The total number of unknowns reduces from d(k+1) to |U|. For these remaining unknowns to be uniquely determined, the rank of the coefficient matrix in Eq. (16) must equal |U|. Since the rank of this coefficient matrix is upperb(k) F (k−1) ), then |U| ≤ rank(A b(k) F (k−1) ). Since |U| + |K| = bounded by rank(A (k+1) d , this yields the following necessary condition for uniquely solving Eq. (16): b(k) F (k−1) . |K| ≥ d(k+1) − rank A

(17)

The original SOE method [4] constructs the linear system at a single input x, which inherently limits the rank of the system to the rank of the local prefix ma(k+1) (k) (k) ⊤ trix F (k−1) around x. However, since the unknown vector is (G1 I S ) , any input point sharing the same activation state in the layers k to r + 1 can be utilized. In other words, as long as the activation states of all neurons in layer k and all subsequent layers remain invariant, the target layer’s activation matrix I (k) and the suffix map G(k+1) are identical across these inputs. Meanwhile, the activation states of neurons in layers 1, . . . , k − 1 may differ across (k−1) these inputs. These differences can produce distinct prefix maps Fxt for xt (1 ≤ t ≤ T ), and hence distinct SOE coefficient matrices. At each xt (1 ≤ t ≤ T ), (1) (1) choose perturbations δt,1 , . . . , δt,d(k+1) that remain in its linear neighborhood. After removing the entries indexed by K, the corresponding SOE systems can be combined as i⊤  b(k) Fx(k−1) δ (1)   A (1) 1 1,1   U (Fθ (x1 + δ1,1 ) − Fθ (x1 ))1   ..     ..   . h   . i⊤      (k−1) (1) (k) (1)   A b   F δ x1  1,d(k+1) U   (Fθ (x1 + δ1,d(k+1) ) − Fθ (x1 ))1    ⊤     (k+1) (k) (k) .. .. . I S =  G1    . .   h   i U ⊤   (1)  (Fθ (xT + δT,1  ) − F (x )) b(k) Fx(k−1) δ (1)   T 1 θ A   T T,1     U .     .. ..       . (1) h i⊤  (Fθ (xT + δT,d(k+1) ) − Fθ (xT ))1 (k−1) (1) b(k) Fx A δ (k+1) 

T,d

(18)

The combined coefficient matrix may have a higher rank than the coefficient matrix obtained at any individual input. When its rank reaches |U|, the remaining unknown entries are uniquely determined.

S. Tang et al.

Hard-Label SOE [3]. Canales-Martı́nez et al. extended the SOE method [4] to S1 setting at LATINCRYPT 2025. Suppose a transition point x ∈ Dab is located at the decision boundary between classes a and b (a < b). According to Eq. (8) and (6), we have: (k+1) (k+1) Dab (x) = G(k+1) − G x(k+1) + γa(k+1) − γb = 0, (19) a b where x(k+1) = I (k) (A(k) (F (k−1) x + β (k−1) ) + b(k) ). The attacker then samples (1) (1) (1) d(k+1) − 1 perturbations δ1 , . . . , δd(k+1) −1 such that x + δℓ ∈ Dab ∩ Lx , with ℓ ∈ [d(k+1) − 1]. Then according to Eq. (19), we have (1) (k+1) b(k) F (k−1) δ (1) = 0. Dab (x + δℓ ) − Dab (x) = G(k+1) − G I (k) S (k) A a b ℓ (20) (1) (1) With d(k+1) − 1 linearly independent equations from δ1 , . . . , δd(k+1) −1 , we can construct the linear system,  h i⊤  b(k) F (k−1) δ (1) A 1   ⊤   (k+1) (k) (k) ..  (G(k+1)  − Gb )I S = 0. (21) a .   i⊤  h (1) b(k) F (k−1) δ (k+1) A d −1 ⊤ (k+1) (k) (k) (k+1) − Gb )I S . Then Solving Eq. (21) yields the unknown vector (Ga the sign recovery process is similar to SOE method introduced above. Limitation. To ensure the linear system in Eq. (21) has a unique nonzero solution up to a scalar multiple, also the same as the SOE method, hard label SOE b(k) F (k−1) ) = d(k+1) , then d(1) , . . . , d(k) ≥ d(k+1) . Consequently, requires rank(A the network architectures also need to be strongly contractive.

Hard-Label Sign Recovery via Normal Alignment

4.1

Recovering Decision-Facet Normals in S5/S1 Settings

Recovering Unit Decision-Facet Normals in the Model Input Space Using [7,5]. point x ∈ Dab , recall from Eq. (19), we have Given a transition (1)

(1)

Dab (x) = Ga − Gb

(1)

x(1) + γa − γb

= 0. The normal vector g (1) of the (1)

(1)

local decision facet Dab ∩ Lx , is given by g (1) := (Ga − Gb )⊤ ∈ Rd . The g (1) can be recovered up to a nonzero scalar using hard-label queries following the methods in [7,5]. Let e1 , . . . , ed(1) denote the standard basis of (1) (1) the model input space Rd . For each ℓ ∈ [d(1) ], let gℓ := ⟨g (1) , eℓ ⟩ denote (1) the ℓ-th coordinate of g (1) . Suppose that g1 ̸= 0. For each ℓ ∈ [d(1) ], take a sufficiently small step αeℓ and use hard-label binary search along e1 to find

Normal Alignment for Sign Recovery on Hard-Label Networks

a scalar βℓ such that x + αeℓ + βℓ e1 returns to the decision boundary Dab . Provided that the returned point remains on the same affine decision facet, then (1) (1) (1) (1) we have αgℓ + βℓ g1 = 0, and hence gℓ /g1 = −βℓ /α. Using these recovered coordinate ratios, we can recover: β (1) g (1) β2 = (1) . (22) gb(1) := 1, − , . . . , − d α α g1 Although its magnitude cannot be recovered, its sign can be determined with the following method. For a sufficiently small ε > 0, x + εb g (1) is still in the linear neighborhood Lx , we have Dab (x + εb g (1) ) − Dab (x) = g (1) · εb g (1) = ε⟨g (1) , gb(1) ⟩.

(23)

– If the hard-label oracle returns class a at x + εb g (1) , then Dab (x + εb g (1) ) − (1) b(1) (1) (1) Dab (x) > 0, i.e., ⟨g , g ⟩ > 0. The sign of gb follows g . – In contrast, if it returns class b, then ⟨g (1) , gb(1) ⟩ < 0, and gb(1) needs to be reversed. After normalization, we obtain the unit decision-facet normal of Dab ∩ Lx as n(1) := g (1) /∥g (1) ∥. Projected Decision-Facet Normal in the Input Space of Layer k. Let g (k) denote the decision-facet normal in the input space of layer k ≥ 1, then h i⊤ (k) (k+1) (k+1) − Gb I (k) A(k) ∈ Rd . Note that the decision-facet g (k) = Ga normal g (1) in the model input space and the decision-facet normal g (k) in the input space of layer k satisfy: g (1) = (F (k−1) )⊤ g (k) .

(24)

This constructs a linear system with an unknown vector g (k) . Then we can recover g (k) from g (1) by solving this system. If rank(F (k−1) ) = d(k) , then the system has the unique solution g (k) . When rank(F (k−1) ) < d(k) , the system admits multiple solutions, and its solution set is g (k) +ker((F (k−1) )⊤ ). Although the full g (k) is not uniquely determined in the latter case, its projection g (k) = P (k) g (k) can be recovered as the unique minimum-norm solution of the system using least squares as proved below. Let h be any solution to Eq. (24), so that h = g (k) + u for some u ∈ (k) ker((F (k−1) )⊤ ). By Def. 7, ker((F (k−1) )⊤ ) = (Vx )⊥ . Since P (k) is the orthog(k) onal projector onto Vx , we have g (k) − P (k) g (k) ∈ ker((F (k−1) )⊤ ). Therefore, we can rewrite h as h = P (k) g (k) + g (k) − P (k) g (k) + u = P (k) g (k) + u′ , (25) (k)

where u′ = g (k) −P (k) g (k) +u ∈ ker((F (k−1) )⊤ ). Since P (k) g (k) ∈ Vx and u′ ∈ (k) (Vx )⊥ , the two vectors are orthogonal. Consequently, ∥h∥2 = ∥P (k) g (k) ∥2 +

S. Tang et al.

∥u′ ∥2 . The norm is therefore uniquely minimized when u′ = 0. Thus, the projected decision-facet normal g (k) = P (k) g (k) is the unique minimum-norm solution of Eq. (24). In the hard-label setting, the attacker recovers the unit decision-facet normal (k) (1) n = g (1) /∥g (1) ∥ rather than g (1) . We therefore solve (F (k−1) )⊤ ∥gg (1) ∥ = n(1) (k)

(k)

and choose any solution ∥gg (1) ∥ ∈ Rd . Following the same argument as above, all such solutions have the same projection onto the control space. Given any (k) (k) solution g (k) /∥g (1) ∥, this common projection is P (k) ∥gg (1) ∥ = ∥gg (1) ∥ . Normalizing this projection gives n(k) :=

4.2

g (k) . ∥g (k) ∥

(26)

Normal Lengths Comparison in Raw-Output Setting (k)

Two-Side Projected Decision-Facet Normals. Let x ∈ Cj ∩ Dab be a dual point, and let Lx and L′x denote the two linear neighborhoods adjacent to the target critical hyperplane. Then the neuron activation states in Lx and L′x differ only in the state of neuron (k, j). Without loss of generality, suppose that the target neuron is active in Lx and inactive in L′x . Denote the corresponding (k) (k) activation matrices by I+ and I− , and the decision-facet normals in the input space of layer k by g (k) and g ′(k) , respectively. Then we have,

h i⊤ (k+1) (k) (k) G(k+1) − G I A a + b X (k+1) (k+1) (k) (k+1) (k+1) (k) = (Ga,i − Gb,i )(Ai )⊤ + (Ga,j − Gb,j )(Aj )⊤ ,

g (k) :=

i∈S/{j}

g ′(k) :=

i⊤ h X (k+1) (k+1) (k) (k) (k) (k+1) A = (Ga,i − Gb,i )(Ai )⊤ , I G(k+1) − G a − b i∈S/{j}

(27) where S contains the indices of all active neurons in layer k. Therefore, the difference between g (k) and g ′(k) is exactly a scalar multiple of the target neuron’s (k) weight Aj , i.e., (k+1)

g (k) − g ′(k) = (Ga,j

(k+1)

− Gb,j

(k)

)(Aj )⊤ .

(28)

According to Eq. (24) in Sect. 4.1, g (k) (or g ′(k) ) can be recovered from the system g (1) = (F (k−1) )⊤ g (k) . In the deeper hidden layers, F (k−1) is often rank deficient, so the full g (k) (or g ′(k) ) is generally not uniquely recoverable. (k) Only their projections onto the control space Vx , g (k) := P (k) g (k) and g ′(k) := (k) ′(k) P g , are uniquely determined.

Normal Alignment for Sign Recovery on Hard-Label Networks

By Def. 7, P (k) is symmetric. Hence, the projected decision-facet normals can be written as h i⊤ (k+1) (k) g (k) := P (k) g (k) = G(k+1) − Gb I+ A(k) P (k) a ⊤ ⊤ X (k+1) (k+1) (k) (k+1) (k+1) (k) = Ga,i − Gb,i P (k) Ai + Ga,j − Gb,j P (k) Aj , i∈S/{j}

′(k)

h i⊤ (k+1) (k) := P (k) g ′(k) = G(k+1) − Gb I− A(k) P (k) a ⊤ X (k+1) (k+1) (k) = Ga,i − Gb,i P (k) Ai . i∈S/{j}

(29)

Statistical Length Advantage. For a matrix M , its squared Frobenius norm P P P (k) 2 is defined by ∥M ∥2F := m ∥Mm ∥2 = m n Mm,n . The matrices I+ A(k) P (k) (k)

and I− A(k) P (k) have identical rows except for the row corresponding to the (k) target neuron, which is Aj P (k) on the active side and zero on the inactive side. Therefore, (k)

I+ A(k) P (k)

2 F

(k)

= I− A(k) P (k)

2 F

(k)

+ Aj P (k)

(30)

which shows that the active side matrix has a larger Frobenius norm, or equivalently, greater total squared row energy. (k+1) (k+1) (k) From Eq. (29), we have g (k) = g ′(k) + (Ga,j − Gb,j )P (k) (Aj )⊤ . Consequently, 2

g (k) − g ′(k) = ⊤ 2 2 ⊤ (k) (k+1) (k+1) (k+1) (k+1) (k) ′(k) (k) . P (k) Aj 2 Ga,j − Gb,j + Ga,j − Gb,j Aj g ,P (31) (k+1)

Generally, the last term in Eq. (31) is positive. If the angle between (Ga,j

−

(k+1) (k) Gb,j )P (k) (Aj )⊤ and g ′(k) is no bigger than π/2, their inner product is non(k) ′(k)

negative. The first term is therefore nonnegative, and ∥g ∥ > ∥g ∥. If the angle is greater than π/2, the first term in Eq. (31) is negative, whereas the last term remains positive. The length ordering is therefore determined by their relative magnitudes. If the last term is larger than the absolute value of the first term, then ∥g (k) ∥ > ∥g ′(k) ∥; otherwise, ∥g (k) ∥ ≤ ∥g ′(k) ∥. Statistical interpretation. We formalize the preceding intuition using an idealized model. (k)

(k)

Proposition 1. We fix I+ A(k) P (k) and I− A(k) P (k) , and treat the suffix co(k+1) (k+1) efficients Ga − Gb as random. Specifically, we assume that their coordinates are independent zero-mean Gaussian variables with common variance

S. Tang et al.

σ 2 > 03 . We have the expectation i h (k) E ∥g (k) ∥2 − ∥g ′(k) ∥2 = σ 2 ∥Aj P (k) ∥2 . (k+1)

(k+1)

(32)

(k+1)

Proof. Let g (k+1) = Ga −Gb with gi denoting its i-th coordinate. For P (k+1) a fixed matrix M , let Mi denote its i-th row. Since g (k+1) M = i gi Mi , bilinearity of the inner product and linearity of expectation give h

E ∥g

(k+1)

M∥

"* X

(k+1) gi Mi ,

(k+1) gj Mj

(33)

(k+1) (k+1) E[gi gj ] ⟨Mi , Mj ⟩ = σ 2

i,j

X ∥Mi ∥2 = σ 2 ∥M ∥2F , i

(k+1) (k+1) gj ] = 0 for i ̸= j, while

since independence and zero means give E[gi (k+1)

E[(gi )2 ] = σ 2 for every i. (k) (k) Replacing M in Eq. (33) by I+ A(k) P (k) and I− A(k) P (k) , and then using Eq. (30), gives Eq. (32). (k)

Thus, whenever Aj P (k) ̸= 0, the active-side projected normal has a larger expected squared length. A positive expected difference alone does not determine how often an individual comparison is correct. Therefore, we introduce the following single-point success probability estimation. Proposition 2 (Single-point success probability). Follow the same asP (k) (k) sumption in Pro. 1 and define τ := i∈S/{j} ⟨Ai P (k) , Aj P (k) ⟩2 ≥ 0, then we have the single-point success probability,   (k) (k) 2 i 1 h ∥A P ∥ 1 j  > 1. (34) Pr ∥g (k) ∥ > ∥g ′(k) ∥ = + arcsin  q (k) (k) 4 2 π 2 ∥Aj P ∥ + 4τ P (k) (k+1) Proof. Define ui := P (k) (Ai )⊤ . According to Eq. (29), g ′(k) = i∈S/{j} gi ui , P P (k+1) (k+1) (k) g = i∈S/{j} gi ui + gj uj , and τ = i∈S/{j} ⟨ui , uj ⟩2 . Eq. (31) can then be written as   X (k+1)  (k+1) (k+1) ∥g (k) ∥2 − ∥g ′(k) ∥2 = gj 2 gi ⟨ui , uj ⟩ + gj ∥uj ∥2  . (35) i∈S/{j}

P (k+1) (k+1) (k+1) Define Z1 := gj and Z2 := 2 i∈S/{j} gi ⟨ui , uj ⟩ + gj ∥uj ∥2 . The active-side projected normal is therefore longer exactly when Z1 Z2 > 0, that is, when Z1 and Z2 have the same sign. 3

Under Kaiming initialization [14], all network weights are independent zero-mean Gaussian variables.

Normal Alignment for Sign Recovery on Hard-Label Networks

(k+1)

By assumption in Pro. 1, the coefficients gi are independent zero-mean Gaussian variables and ui are fixed. For any α, β ∈ R, αZ1 + βZ2 is a linear (k+1) combination of gi and is therefore Gaussian. Hence, (Z1 , Z2 ) is a jointly Gaussian pair. Their means are zero by the linearity of expectation. (k+1) Since Z1 = gj , we immediately have Var(Z1 ) = σ 2 . For Z2 , we have   X (k+1) (k+1) gi ⟨ui , uj ⟩ + gj ∥uj ∥2  Var(Z2 ) = Var 2 i∈S/{j}

(k+1)

⟨ui , uj ⟩ Var(gi

(k+1)

) + ∥uj ∥4 Var(gj

) = σ 2 4τ + ∥uj ∥4 .

i∈S/{j}

(36) Since Z1 and Z2 have zero mean, their covariance is Cov(Z1 , Z2 ) = E[Z1 Z2 ]. (k+1) (k+1) (k+1) (k+1) For i ∈ S/{j}, the independence gives E[gj gi ] = E[gj ]E[gi ] = 0. Therefore, 





(k+1) 

Cov(Z1 , Z2 ) = E gj

(k+1)

⟨ui , uj ⟩ + gj

∥uj ∥2 

i∈S/{j}

(37)

(k+1) 2 (k+1) (k+1) ) ] = σ 2 ∥uj ∥2 . ] + ∥uj ∥2 E[(gj gi ⟨ui , uj ⟩ E[gj

i∈S/{j}

Using Var(Z1 ) = σ 2 , Var(Z2 ) = σ 2 (4τ + ∥uj ∥4 ), and Cov(Z1 , Z2 ) = σ 2 ∥uj ∥2 , the correlation coefficient between Z1 and Z2 is Corr(Z1 , Z2 ) = p

∥uj ∥2 . =p ∥uj ∥4 + 4τ Var(Z1 ) Var(Z2 ) Cov(Z1 , Z2 )

(38)

Since uj ̸= 0, Corr(Z1 , Z2 ) > 0. Thus, Z1 and Z2 are positively correlated. Dividing Z1 and Z2 by their positive standard deviations does not change their signs or their correlation coefficient. The resulting variables form a standard jointly Gaussian pair. For such a pair, the standard Gaussian quadrant identity gives Pr[Z1 > 0, Z2 > 0] = 1/4 + arcsin(Corr(Z1 , Z2 ))/(2π). Moreover, the zeromean jointly Gaussian distribution is centrally symmetric, so Pr[Z1 < 0, Z2 < 0] = Pr[Z1 > 0, Z2 > 0]. By Eq. (35), the active-side projected normal is longer exactly when Z1 Z2 > 0. Therefore, h i 1 1 Pr ∥g (k) ∥ > ∥g ′(k) ∥ = Pr[Z1 Z2 > 0] = + arcsin (Corr(Z1 , Z2 )) 2 π ! (39) 1 1 ∥uj ∥2 = + arcsin p . 2 π ∥uj ∥4 + 4τ Since Corr(Z1 , Z2 ) > 0, the probability in Eq. (39) is strictly greater than 1/2. (k) Finally, substituting uj = P (k) (Aj )⊤ proves Eq. (34). P (k) (k) (k) Proposition 3. The τ := , Aj P (k) ⟩2 is the sum of the i∈S/{j} ⟨Ai P squared inner products between the projected weight of the target neuron (neuron

S. Tang et al.

j) and the projected weights of the other active neurons (neuron i ∈ S/{j}). It (k) therefore measures their total alignment with the target direction Aj P (k) . A larger alignment τ generally reduces the single-point success probability. 4.3

Normal Alignment in Hard-Label Setting

The preceding analysis gives the length comparison: according to Eq. (34), the side with the longer g (k) is predicted to be the target active side with a higher probability. However, the hard-label queries recover only the unit projected normals n(k) and n′(k) as stated in the last paragraph of Sect. 4.1, thereby losing the lengths of g (k) and g ′(k) . Consequently, the length comparison cannot be applied directly. We therefore compare the absolute alignments of the two unit b(k) . projected normals with the recovered target weight A j Proposition 4 (Equivalence of the length and alignment comparisons). b(k) )⊤ ̸= 0, and P (k) (A b(k) )⊤ ∦ g ′(k) . Then Assume g (k) ̸= 0, g ′(k) ̸= 0, P (k) (A j j ⊤ ⊤ ′(k) b(k) b(k) > . (40) A g (k) > g ′(k) n(k) , A n , ⇐⇒ j j Proof. According to Eq. (26), we have P (k) n(k) = n(k) and P (k) n′(k) = n′(k) . b(k) )⊤ /∥P (k) (A b(k) )⊤ ∥. Using these identities and the symDefine m := P (k) (A j j metry of P (k) , we obtain ⊤ ⊤ ⊤ ⊤ D E b(k) b(k) b(k) b(k) n(k) , A = P (k) n(k) , A = n(k) , P (k) A = P (k) A n(k) , m , j j j j ⊤ D ⊤ ⊤ ⊤ E (k) ′(k) ′(k) (k) b(k) b(k) b(k) b(k) n′(k) , A = P n , A = n , P A = P (k) A n′(k) , m . j j j j

(41) (k) ⊤ b The common factor ∥P (Aj ) ∥ is positive and therefore does not affect the ordering of the absolute inner products. b(k) and A(k) differ only by a nonzero scalar, Eq. (29) shows that g (k) − Since A j j g ′(k) is parallel to m. Hence, there exist α, β ∈ R and a vector m⊥ ⊥ m such that g ′(k) = αm + m⊥ and g (k) = βm + m⊥ . Thus, they share the same orthogonal component m⊥ . Since m is a unit vector and m⊥ ⊥ m, we have ∥g ′(k) ∥2 = α2 + ∥m⊥ ∥2 and ∥g (k) ∥2 = β 2 + ∥m⊥ ∥2 . Using n(k) = g (k) /∥g (k) ∥ and n′(k) = g ′(k) /∥g ′(k) ∥, we obtain |⟨n(k) , m⟩|2 = β 2 /(β 2 + ∥m⊥ ∥2 ) and |⟨n′(k) , m⟩|2 = α2 /(α2 + ∥m⊥ ∥2 ). Consequently, D E2 D E2 n(k) , m − n′(k) , m β 2 (α2 +∥m⊥ ∥2 )−α2 (β 2 +∥m⊥ ∥2 ) β2 α2 (42) = β 2 +∥m − = 2 2 2 α +∥m⊥ ∥ (β 2 +∥m⊥ ∥2 )(α2 +∥m⊥ ∥2 ) ⊥∥ 2 2 2 2 ∥m⊥ ∥ (β −α ) ⊥∥ = (β 2 +∥m⊥ ∥2 )(α2 +∥m⊥ ∥2 ) = ∥g(k)∥m ∥g (k) ∥2 − ∥g ′(k) ∥2 . ∥2 ∥g ′(k) ∥2 (k)

b(k) )⊤ ̸= 0, and P (k) (A b(k) )⊤ ∦ With the assumptions g (k) ̸= 0, g ′(k) ̸= 0, P (k) (A j j g ′(k) , the factor ∥m⊥ ∥2 /(∥g (k) ∥2 ∥g ′(k) ∥2 ) is strictly positive. Hence, ∥g (k) ∥ >

Normal Alignment for Sign Recovery on Hard-Label Networks

∥g ′(k) ∥ if and only if |⟨n(k) , m⟩| > |⟨n′(k) , m⟩|. By Eq. (41), the latter is equivb(k) )⊤ ⟩| > |⟨n′(k) , (A b(k) )⊤ ⟩|, which proves Pro. 4. alent to |⟨n(k) , (A j j According to Pro. 4, the alignment comparison has the same single-point success probability given in Eq. (34). Based on this comparison, we introduce the sign recovery method Normal Alignment for hard-label networks. Proposition 5 (Normal Alignment). At each dual point, the side whose unit projected decision-facet normal n(k) has the larger absolute inner product with b(k) )⊤ is predicted to be the target active side. If this prediction agrees with the (A j active side indicated by the recovered signature, the dual point votes to retain the b(k) ; otherwise, it votes to reverse its sign. sign of A j Normal Alignment repeats this comparison at multiple dual points and aggregates the resulting votes. The majority vote determines whether the sign of the recovered signature is retained or reversed. The fraction of valid votes supporting this decision is used as its confidence level. For a target neuron, let n+ and n− denote the numbers of votes for retaining and reversing the recovered signature, respectively, and let nvalid := n+ + n− . For nvalid > 0, define the confidence level as α := max{n+ , n− }/nvalid . Given a confidence threshold α0 ∈ (1/2, 1], the Normal Alignment retains the recovered signature if n+ > n− and α ≥ α0 , and reverses it if n− > n+ and α ≥ α0 . Otherwise, the available votes are insufficient to determine the sign, which remains unresolved. White-Box Validation of the Normal Alignment. We experimentally validate the Normal Alignment in the white-box setting on the CIFAR-10 DNNs with architectures 192-d×3-10 for d ∈ {32, 64, 128, 256}. As shown in Fig. 3 in Supp. A, nattempt = 200 provides high sign recovery accuracy. Table 3 summarizes the layer-wise results. For each dual point, the projected normal length ∥g (k) ∥ comparison in Eq. (31) is quantified by ∥g ′(k) ∥ . The corresponding column reports the median of this ratio over all evaluated dual points in each layer. For each neuron in a layer, Eq. (34) is used to compute a theoretical success probability at each of its dual points and “pth ” is the mean of these probabilities over all evaluated dual points in the layer. Correspondingly, pobs is the proportion of correct single-point votes among all evaluated dual points in that layer. The column of “Agreement” reports the percentage of evaluated dual points in each layer satisfying the equivalence in Eq. (40) of Pro. 4. The column of “Signs recovered” gives the number of correctly recovered neuron signs in each layer after aggregating 200 votes per neuron. ∥g (k) ∥ As shown in Table 3, the median of the ratio ∥g ′(k) ∥ is greater than one in all twelve hidden layers. For all the evaluated dual points of the four DNNs, the mean single-point success probabilities are pth = 66.01% theoretically and pobs = 70.12% empirically. Thus, the theoretical model captures the advantage over random guessing, although it underestimates its magnitude. Consistent with Pro. 4, the length and alignment rules agree (Eq. (40) is satisfied) on

S. Tang et al.

Table 3: Layer-wise white-box validation of Normal Alignment on CIFAR-10 DNNs 192-d-d-d-10 using nattempt = 200 dual points per neuron. d

Layer k

∥g (k) ∥/∥g ′(k) ∥

pth /pobs (%)

Agreement (%)

Signs recovered

1 2 3

1.014 1.010 1.016

74.94/76.61 64.31/66.92 66.86/69.56

100.00 100.00 100.00

32/32 30/32 31/32

1 2 3

1.008 1.007 1.012

71.04/74.68 65.12/69.12 66.25/71.84

100.00 100.00 100.00

64/64 64/64 62/64

128

1 2 3

1.004 1.004 1.004

68.04/73.15 65.62/68.79 65.40/68.82

100.00 100.00 100.00

128/128 128/128 126/128

256

1 2 3

1.002 1.002 1.002

64.99/70.81 66.00/68.95 64.41/68.73

100.00 100.00 100.00

256/256 255/256 241/256

every evaluated dual point. After vote aggregation, Normal Alignment correctly recovers 1417 of the 1440 neuron signs, giving an aggregate recovery accuracy of 1417 1440 × 100% = 98.40%. In particular, all signs in the first hidden layer are recovered correctly for all four models. Meanwhile, the second and third hidden layers contain 20 incorrectly recovered signs and 3 tied outcomes.

eSOE+Alignment: Combining Normal Alignment with Hard-Label SOE

At NeurIPS 2024, Foerster et al. [13] empirically observed that many neuron signs recovered by Neuron Wiggle [4] remained at low confidence and that collecting additional critical points did not improve their confidence. Therefore, they performed an exhaustive search on these low-confidence signs, which led to a significant increase in the number of model queries and runtime, even turning the so-called polynomial-time attack into an exponential-time attack. As a probabilistic voting method, the Normal Alignment may also leave some neuron signs undetermined when their voting confidence is insufficient. In contrast, Hard-label SOE [3] can deterministically recover the activation states of all neub(k) F (k−1) ) = d(k+1) to solve the linear rons in layer k, but it requires rank(A system in Eq. (21). Inspired by the raw-output SOE+Wiggle [18], we proposed hard-label eSOE+Alignment for high-confidence sign recovery. Recall from Sect. 3.2, SOE + Wiggle contains two main parts: removing inactive neurons identified with high-confidence level in Eq. (16) and extending the system from multiple points in Eq. (18). In S1 access, removing inactive neurons remains straightforward: at a selected transition point, the signs recovered by Normal Alignment with high confidence are used to identify inactive neurons, and their corresponding zero entries are removed from the hard-label SOE Eq. (21) in Sect. 3.2. The system extension, however, cannot be applied directly.

Normal Alignment for Sign Recovery on Hard-Label Networks

Finding Compatible Transition Points. According to Eq. (21), the un ⊤ (k+1) (k+1) (k) (k) known vector is (Ga − Gb )I S . To extend the linear system in Eq. (21), the selected points must share the same unknowns. Therefore, two conditions should be simultaneously satisfied: – First, the selected points must share the activation states in layer k and all subsequent layers; otherwise, their corresponding hard-label SOE systems in Eq. (21) have different unknown vectors; – Second, the points should remain on the decision boundary between the same two classes, i.e., Dab . We can only walk along the decision boundary to ensure that no future neurons have toggled, and check the output label from slightly perturbing Fθ (xt + δ (1) ) and Fθ (xt − δ (1) ) to keep xt ∈ Dab . We use the idea for locating dual points introduced in [5] to find compatible transition points on the decision boundary between fixed two classes, specifically, – Step 1: From x1 ∈ Dab ∩ Lx1 , the attacker makes a random excursion and uses hard-label binary search to relocate another point x′1 ∈ Dab ∩Lx1 . Then the difference x′1 − x1 determines a direction along Dab ∩ Lx1 . Following the direction of x′1 − x1 , the attacker walks until the decision boundary bends at a dual point x2 (suppose that x1 ∈ Dab ∩ Lx2 ). – Step 2: Since the parameters of layers 1, . . . , k − 1 have already been recovered, the attacker can evaluate the pre-activation values of the neurons in these layers at x2 . If a neuron in a preceding layer is zero before ReLU, the bend is attributed to that layer; the attacker then makes a random excursion and uses binary search to locate another x′2 on the adjacent decision facet Dab ∩ L′x2 . Otherwise, the bend may be caused by a neuron in layer k or a subsequent layer, so the attacker terminates the current search path. – Step 3: If the bend is attributed to a neuron in a preceding layer,the attacker then collects x′2 for SOE extension and continues to find the next decision boundary bend along the direction of x′2 − x2 . Repeating this procedure from the starting point x1 yields sufficient transition points for the following extension. Scale-Free Hard-label SOE Extension. Let x1 , . . . , xT be the collected transition points. Since these points share the same activation matrices in layer k (1) and all subsequent layers, their I (k) and G(k+1) are identical. Let gt be the (1) (1) (1) (1) normal of the decision facet Dab ∩ Lxt ⊂ Rd , and nt = gt /∥gt ∥. The decision-facet normal is often already available from signature recovery and can therefore be reused here, avoiding the additional oracle queries needed to search for perturbations and construct the equations in Eq. (21). According to Eq. (24) (1) (k+1) (k+1) ⊤ in Sect. 4.1, gt = (A(k) Fxk−1 )⊤ I (k) (Ga − Gb ) . With the recovered t

S. Tang et al. (k−1)

b(k) and prefix map Fxt signature A

, we have ⊤ S (k) ⊤ (1) (k+1) (k+1) (k) b(k) F (k−1) nt = A G − G . I a xt b (1) ∥gt ∥ (k+1)

(43)

(k+1)

Although S (k) I (k) (Ga − Gb )⊤ is common to all selected points, the (1) 1/∥gt ∥ generally differs across them. Therefore, thesehard-label SOE systems (1)

(1)

(1)⊤

cannot be combined directly. Since nt is a unit vector, Idd(1) − nt nt

(1)

0, where Idd(1) is the d(1) ×d(1) identity matrix. Multiplying both sides of Eq. (43) (1) (1)⊤ (1) by Idd(1) − nt nt to remove 1/∥gt ∥, and it gives ⊤ ⊤ (k+1) (1) (1)⊤ (k+1) (k) (k) (k) (k−1) b Ga − Gb Idd(1) − nt nt S I = 0. A Fxt (44) Now, the unknown vector in Eq. (44) is the same for all selected transition points. Again, let K contain the neurons that Normal Alignment identifies as inactive in layer k at xt with high confidence, and U := [d(k+1) ]\K contain the remaining neurons. For a vector v, [v]U denotes the subvector indexed by U. Restricting Eq. (44) to U and combining the equations from all collected transition points, we obtain  ⊤  (1) (1)⊤ b(k) Fx(k−1) [ Idd(1) − n1 n1 A ]U 1  ⊤    [ Id (1) − n(1) n(1)⊤ A ⊤ b(k) Fx(k−1) ]U  2 d   (k) (k) 2 2 (k+1) = 0. S I G(k+1) − G   a b ..   U   .  ⊤  (1) (1)⊤ b(k) Fx(k−1) [ Idd(1) − n n A ]U T T

(45) Proposition 6 (eSOE+Alignment). Assume every i h neuron in K is inactive (k+1)

at the selected transition points and the entries of (Ga

(k+1) ⊤

− Gb

)

cor-

responding to active neurons are nonzero. If the coefficient matrix in Eq. (45) has rank |U| − 1, then its nonzero solution is uniquely determined up to a scalar multiple. Then, eSOE+Alignment recovers all the neuron signs in layer k in polynomial time. The correctness of the sign assignment returned by eSOE+Alignment can be assessed by attempting signature recovery for the next layer, as proposed by [13]. If the next-layer signatures cannot be recovered, eSOE may have incorrectly eliminated a column corresponding to a neuron that is actually active at the selected transition points, making the solution unreliable.We therefore discard the signs returned by eSOE+Alignment and instead use the signs predicted by Normal Alignment. We then apply the exhaustive-search strategy of [13] to enumerate candidate assignments for the low-confidence signs until the next-layer signatures are successfully recovered. We handle eSOE+Toggle analogously: when its

Normal Alignment for Sign Recovery on Hard-Label Networks

reduced SOE solution is unreliable, we apply the same exhaustive-search strategy to the low-confidence signs predicted by Future Toggle. We validate eSOE+Alignment on the CIFAR-10 model with architecture 3072-256×3-64-10. Following the proof-of-concept setting in [5], we use exact decision-facet normals and set nattempt = 200. For every hidden layer, the projected coefficient matrix satisfies the rank condition in Proposition 6, and eSOE+Alignment correctly recovers all 832 signs, as reported in Table 4 in Supp. A. The same result is observed in Table 1, where eSOE+Alignment recovers all 512 and 320 signs, respectively. Thus, no fallback exhaustive search is required in any of the evaluated settings. For the 3072-256×3-64-10 model, complete sign recovery takes about 1h43m on our 8-core CPU without neuron-level parallelism. For reference, Carlini et al. [5] reported an estimated runtime of 8.5 hours for complete sign recovery using Future Toggle on a 256-core server, with 64 neurons processed in parallel.

Experiments

We evaluate sign recovery on two trained ReLU DNNs: a CIFAR-10 model with architecture 192-64×8-10 and an MNIST model with architecture 64-96×3-32-10, containing 42,122 and 28,298 parameters, respectively.4 Both networks use ReLU activations and Kaiming normal initialization [14]. For the target layer k, the sign recoveries are evaluated assuming that the preceding layers have been recovered and the layer k’s neuron signatures are known up to sign. Following the experimental convention of [5], we assume that the dual points have been precomputed because this step is shared by signature recovery and sign recovery. We use 200 attempted dual points per neuron for Normal Alignment and eSOE+Alignment, and evaluate Future Toggle and eSOE+Toggle with budgets of both 200 and 1000. For CIFAR-10, we follow the proof-of-concept setting of [5]. Decision-facet normals are computed directly from model parameters. Future Toggle additionally uses white-box information to walk along the decision boundary and handle non-future toggles. For MNIST, we recover the unit decision-facet normals using the procedure described in Sect. 4.1, and Future Toggle discards a dual point if the boundary walk first encounters a nonfuture toggle. Tables 5, 6, and 7 in Supp. A summarize the results across all hidden layers. eSOE+Alignment correctly recovers all 512 and 320 signs in the two models, respectively.

Conclusion

This paper proposes Normal Alignment, an improved statistical sign-recovery method for hard-label ReLU network extraction to address the limitations of Future Toggle. Using projected decision-facet normals at dual points, it achieves 4

CIFAR-10 and MNIST images are resized and flattened to 8 × 8 × 3 = 192 and 8 × 8 = 64 dimensions, respectively.

S. Tang et al.

higher voting accuracy and pushes errors to low-confidence ranks. Combined with extended hard-label SOE, eSOE+Alignment achieves exact polynomial-time full sign recovery without exponential enumeration. Evaluations on CIFAR-10 and MNIST confirm its superiority.

References 1. Asselineau, R., Derbez, P., Fouque, P., Minaud, B.: Cryptanalytic extraction of deep neural networks with non-linear activations. In: Heninger, N., Rosulek, M. (eds.) Advances in Cryptology - CRYPTO 2026 - 46th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 17-20, 2026, Proceedings, Part VII. Lecture Notes in Computer Science, vol. 16806, pp. 36–66. Springer (2026). https://doi.org/10.1007/978-3-032-35415-0 2, https://doi.org/10.1007/ 978-3-032-35415-0_2 2. Batina, L., Bhasin, S., Jap, D., Picek, S.: CSI NN: Reverse engineering of neural network architectures through electromagnetic side channel. In: 28th USENIX Security Symposium (USENIX Security 19). pp. 515–532 (2019) 3. Canales-Martı́nez, I.A., Santos, D.: Extracting some layers of deep neural networks in the hard-label setting. In: Escudero, D., Damgård, I. (eds.) Progress in Cryptology - LATINCRYPT 2025 - 9th International Conference on Cryptology and Information Security in Latin America, Medellı́n, Colombia, October 1-3, 2025, Proceedings. Lecture Notes in Computer Science, vol. 16129, pp. 399–421. Springer (2025). https://doi.org/10.1007/978-3-032-06754-8 15, https: //doi.org/10.1007/978-3-032-06754-8_15 4. Canales-Martı́nez, I.A., Chávez-Saab, J., Hambitzer, A., Rodrı́guez-Henrı́quez, F., Satpute, N., Shamir, A.: Polynomial time cryptanalytic extraction of neural network models. In: Joye, M., Leander, G. (eds.) Advances in Cryptology - EUROCRYPT 2024 - 43rd Annual International Conference on the Theory and Applications of Cryptographic Techniques, Zurich, Switzerland, May 26-30, 2024, Proceedings, Part III. Lecture Notes in Computer Science, vol. 14653, pp. 3–33. Springer (2024). https://doi.org/10.1007/978-3-031-58734-4 1, https: //doi.org/10.1007/978-3-031-58734-4_1 5. Carlini, N., Chávez-Saab, J., Hambitzer, A., Rodrı́guez-Henrı́quez, F., Shamir, A.: Polynomial time cryptanalytic extraction of deep neural networks in the hard-label setting. In: Fehr, S., Fouque, P. (eds.) Advances in Cryptology - EUROCRYPT 2025 - 44th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Madrid, Spain, May 48, 2025, Proceedings, Part I. pp. 364–396. Lecture Notes in Computer Science, Springer (2025). https://doi.org/10.1007/978-3-031-91107-1 13, https:// doi.org/10.1007/978-3-031-91107-1_13 6. Carlini, N., Jagielski, M., Mironov, I.: Cryptanalytic extraction of neural network models. In: Micciancio, D., Ristenpart, T. (eds.) Advances in Cryptology - CRYPTO 2020 - 40th Annual International Cryptology Conference, CRYPTO 2020, Santa Barbara, CA, USA, August 17-21, 2020, Proceedings, Part III. pp. 189–218. Lecture Notes in Computer Science, Springer (2020). https://doi.org/10.1007/978-3-030-56877-1 7, https://doi.org/10.1007/ 978-3-030-56877-1_7 7. Chen, Y., Dong, X., Guo, J., Shen, Y., Wang, A., Wang, X.: Hard-label cryptanalytic extraction of neural network models. In: Chung, K., Sasaki, Y. (eds.)

Normal Alignment for Sign Recovery on Hard-Label Networks

Advances in Cryptology - ASIACRYPT 2024 - 30th International Conference on the Theory and Application of Cryptology and Information Security, Kolkata, India, December 9-13, 2024, Proceedings, Part VIII. pp. 207–236. Lecture Notes in Computer Science, Springer (2024). https://doi.org/10.1007/978-981-96-0944-4 7, https://doi.org/10.1007/978-981-96-0944-4_7 8. Chen, Y., Dong, X., Ma, R., Shen, Y., Wang, A., Yu, H., Wang, X.: Delving into cryptanalytic extraction of prelu neural networks. In: International Conference on the Theory and Application of Cryptology and Information Security. pp. 576–607. Springer (2025) 9. Chen, Z., Tang, S., Gao, Z., Su, Y., Qin, L., Dong, X.: Algebraic attack on convolutional neural networks with max pooling. In: Heninger, N., Rosulek, M. (eds.) Advances in Cryptology - CRYPTO 2026 - 46th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 17-20, 2026, Proceedings, Part VII. Lecture Notes in Computer Science, vol. 16806, pp. 3–35. Springer (2026). https://doi.org/10.1007/978-3-032-35415-0 1, https://doi.org/10.1007/ 978-3-032-35415-0_1 10. Chen, Z., Tang, S., Gao, Z., Su, Y., Qin, L., Dong, X.: Algebraic cryptanalytic extraction on hard-label neural networks. Cryptology ePrint Archive, Paper 2026/1164 (2026), https://eprint.iacr.org/2026/1164 11. Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017). https://doi.org/10.1038/nature21056, https: //doi.org/10.1038/nature21056 12. Fefferman, C., et al.: Reconstructing a neural net from its output. Revista Matemática Iberoamericana 10(3), 507–556 (1994) 13. Foerster, H., Mullins, R., Shumailov, I., Hayes, J.: Beyond slow signs in highfidelity model extraction. In: Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J.M., Zhang, C. (eds.) Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024 (2024), http://papers.nips.cc/paper_files/paper/2024/hash/ 22ae669a35bb9e70eb93ab77c1eff5b4-Abstract-Conference.html 14. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015. pp. 1026–1034. IEEE Computer Society (2015). https://doi.org/10.1109/ICCV.2015.123, https://doi.org/10.1109/ICCV.2015. 123 15. Ito, A., Miura, T., Todo, Y.: Is the hard-label cryptanalytic model extraction really polynomial? In: Heninger, N., Rosulek, M. (eds.) Advances in Cryptology CRYPTO 2026 - 46th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 17-20, 2026, Proceedings, Part VII. Lecture Notes in Computer Science, vol. 16806, pp. 67–98. Springer (2026). https://doi.org/10.1007/978-3-03235415-0 3, https://doi.org/10.1007/978-3-032-35415-0_3 16. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems. vol. 25. Curran Associates, Inc. (2012), https://proceedings.neurips.cc/paper_files/paper/ 2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf

S. Tang et al.

17. Liu, H., Siproudhis, A., Boura, C., Peyrin, T.: Model extraction of convolutional neural networks with max-pooling. Cryptology ePrint Archive (2026) 18. Liu, H., Siproudhis, A., Experton, S., Lorenz, P., Boura, C., Peyrin, T.: Navigating the deep: End-to-end extraction on deep neural networks. In: Daemen, J., Thomé, E. (eds.) Advances in Cryptology - EUROCRYPT 2026 - 45th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Rome, Italy, May 10-14, 2026, Proceedings, Part VI. Lecture Notes in Computer Science, vol. 16546, pp. 482–512. Springer (2026). https://doi.org/10.1007/978-3-032-253330 17, https://doi.org/10.1007/978-3-032-25333-0_17 19. Lowd, D., Meek, C.: Adversarial learning. In: Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining. pp. 641–647 (2005) 20. Oliynyk, D., Mayer, R., Rauber, A.: I know what you trained last summer: A survey on stealing machine learning models and defences. ACM Computing Surveys 55(14s), 1–41 (2023) 21. Qi, X., Lei, H., Wei, L., Sun, X., Wang, M.: Cryptanalytic extraction of neural networks with various activation functions. Cryptology ePrint Archive (2026) 22. Sun, X., Lei, H., Wei, L., Qi, X., Hu, K., Wang, M., Wang, W.: Cryptanalytic extraction of convolutional neural networks. Cryptology ePrint Archive, Paper 2026/139 (2026), https://eprint.iacr.org/2026/139 23. Tramèr, F., Zhang, F., Juels, A., Reiter, M.K., Ristenpart, T.: Stealing machine learning models via prediction APIs. In: 25th USENIX security symposium (USENIX Security 16). pp. 601–618 (2016) 24. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems 30. pp. 5998–6008. Curran Associates, Inc. (2017), https: //proceedings.neurips.cc/paper/7181-attention-is-all-you-need 25. Wei, L., Lei, H., Qi, X., Sun, X., Gao, L., Hu, K., Wang, W., Wang, M.: Cryptanalytic extraction of recurrent neural network models. Cryptology ePrint Archive (2026)

Normal Alignment for Sign Recovery on Hard-Label Networks

Supplementary Material A

Supporting Experimental Results

&RUUHFWO\UHFRYHUHGQHXURQV

d = 32 d = 64 d = 128 d = 256

nattempt

Fig. 3: Sign recovery accuracy of Normal Alignment. Each curve reports the percentage of correctly recovered signs across all three hidden layers after aggregating the nattempt votes for each neuron.

Table 4: Sign recovery results of Normal Alignment and eSOE+Alignment on the CIFAR-10 model with architecture 3072-256×3-64-10. Metric

All hidden layers

Correct signs Normal Alignment Vote accuracy p (%)

256/256 77.85

256/256 72.09

255/256 67.85

63/64 74.46

830/832 72.74

Correct signs eSOE+ Alignment Projected rank Min confidence

256/256 256/256 256/256 64/64 255 146 64 63 n/a 0.73 0.60 n/a

832/832 – 0.60

Method

Projected rank: The rank of the coefficient matrix in Eq. (45) after projection and column elimination. In every layer, the reported rank equals |U| − 1, so the nonzero solution is uniquely determined up to a scalar multiple. Min confidence: The minimum Normal Alignment confidence used to eliminate inactive-neuron columns. N/A indicates that the unprojected stacked coefficient matrix already has full column rank, so no column elimination is required before projection.

S. Tang et al.

Table 5: Layer-wise confidence and ranks of highest-confidence incorrect sign predictions produced by Normal Alignment and Future Toggle [5] on the CIFAR10 and MNIST models. Model

CIFAR-10 192-64×8-10

Hidden Layer

Method

Metric L1

Normal Alignment nattempt = 200

I-Confidence (%) I-Rank

✓

50.50 51.00 51.50 55.50 64.00 55.50 55.50 64/64 64/64 61/64 60/64 50/64 60/64 58/64

Future Toggle nattempt = 200

I-Confidence (%) I-Rank

✓

51.05 53.01 54.55 56.89 55.70 58.72 100.00 61/64 42/64 28/64 15/64 26/64 12/64 1/64

Future Toggle nattempt = 1000

I-Confidence (%) I-Rank

✓

Normal Alignment nattempt = 200

I-Confidence (%) 55.43 62.30 60.11 56.99 I-Rank 88/96 68/96 78/96 29/32

–

I-Confidence (%) 50.56 60.71 80.00 100.00 I-Rank 96/96 27/96 4/96 1/32

–

57.46 63.77 100.00† 49/96 15/96 1/32

–

MNIST Future Toggle 64-96×3-32-10 nattempt = 200 Future Toggle nattempt = 1000

I-Confidence (%) I-Rank

✓

50.62 51.90 53.36 55.58 70.00 60/64 44/64 45/64 20/64 13/64

I-Confidence: It is the highest confidence among all the incorrect sign predictions. Therefore, the sign-recovery method with lower I-Confidence is better. I-Rank: Confidence ranks are computed in descending order among all neurons in the corresponding layer; rank 1 denotes the highest confidence. I-Rank is the confidence rank of the highest-confidence incorrect sign prediction. This metric is critical for confidence-ordered enumeration in complete sign recovery: following [13], all signs at or below the I-Rank in the confidence ordering, including the sign at the I-Rank itself, must be included in the enumeration. Therefore, a method is better when its highest-confidence error occurs lower in the confidence ordering, i.e., at a larger numerical I-Rank. ✓: No incorrect sign prediction is produced in this layer. †: Dual-point utilization in the final hidden layer is only 0.21%; there is a neuron with only two valid votes, both of which are incorrect.

Metric

✓

0.64

✓

0.68

0.60

✓

64/64 70.30 100.00 25.55 (s) 228.15

0.66

✓

64/64 71.05 100.00 29.43 (s) 228.15

512/512 70.70 100.00 10.73 2 (s) 231.13 – – 0.60

✓

0.55

✓

0.56

0.53

64/64 54.24 87.43 29.60 (s) 231.92

✓

54/64 53.59 80.21 9.86 2 (s) + 221 (g) 232.13 51.90 44/64 0.52

0.54

✓

64/64 54.42 71.40 10.20 2 (s) 232.41

55/64 40/64 469/512 53.40 56.89 62.45 55.98 2.20 73.00 10.83 45 11.98 52 13.47 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 252 (g) 232.92 233.87 235.49 55.58 70.00 – 20/64 13/64 – 0.52 0.58 0.52

61/64 59/64 57/64 54/64 47/64 470/512 54.24 53.59 54.42 53.40 56.89 62.45 87.43 80.21 71.40 55.98 2.20 73.00 9.24 5 9.54 21 9.89 20 10.5 45 11.66 52 13.17 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 252 (g) 231.92 232.13 232.41 232.92 233.87 235.49 50.62 51.90 53.36 55.58 70.00 – 60/64 44/64 45/64 20/64 13/64 –

64/64 55.34 91.69 29.35 (s) 231.76

✓

64/64 56.42 95.36 29.17 (s) 231.64

64/64 55.34 91.69 28.98 (s) 231.76

64/64 56.42 95.36 28.80 (s) 231.64

51/64 47/64 49/64 50/64 36/64 425/512 53.96 53.45 54.18 53.77 56.40 62.29 87.38 80.84 71.90 56.04 2.26 73.10 28.15 (s) + 237 (g) 28.31 (s) + 250 (g) 28.66 (s) + 239 (g) 29.26 (s) + 253 (g) 29.56 (s) + 264 (g) 211.65 (s) + 264 (g) 229.60 229.81 230.08 230.59 231.57 233.17 54.55 56.89 55.70 58.72 100.00 – 28/64 15/64 26/64 12/64 1/64 – 0.53 0.51 0.53 0.51 0.60 0.51

I-Confidence/I-Rank: These metrics are defined in Table 5. A ✓ indicates that no incorrect sign prediction is produced in the corresponding layer. If a method combined with eSOE does not recover all signs in a layer, we report the I-Confidence and I-Rank of its underlying statistical method, Normal Alignment or Future Toggle, because the fallback exhaustive search uses that method’s confidence ordering. Min confidence: The minimum confidence used by eSOE to eliminate inactive-neuron columns. N/A indicates that the unprojected stacked coefficient matrix already has full column rank, so no column elimination is required before projection. Time: 2x (s) denotes the method runtime in seconds, whereas 2y (g) denotes the estimated number of candidate sign assignments required by confidence-ordered exhaustive search for complete recovery. The guessing term is omitted when all signs are correctly recovered.

Correct signs 64/64 Vote accuracy p (%) 100.00 ηdual (%) 99.78 eSOE+ Time 210.41 (s) Toggle 231.44 nattempt = 1000 Queries I-Confidence (%) ✓ I-Rank Min confidence n/a

Correct signs 64/64 Vote accuracy p (%) 100.00 ηdual (%) 99.78 Future Toggle Time 210.28 (s) nattempt = 1000 Queries 231.44 I-Confidence (%) ✓ I-Rank

✓

0.53

✓

64/64 54.61 91.73 27.93 (s) 229.44

0.56

64/64 56.59 94.86 27.77 (s) 229.33

eSOE+ Toggle nattempt = 200

0.64

✓

64/64 70.33 100.00 27.82 (s) 228.15

Correct signs 64/64 Vote accuracy p (%) 100.00 ηdual (%) 99.78 Time 28.55 (s) Queries 229.12 I-Confidence (%) ✓ I-Rank Min confidence n/a

0.63

✓

64/64 70.73 100.00 27.48 (s) 228.13

495/512 70.70 100.00 6.17 2 (s) + 215 (g) 231.13 – –

All hidden layers

Future Toggle nattempt = 200

0.61

64/64 68.46 100.00 27.40 (s) 228.13

64/64 69.65 100.00 27.18 (s) 228.12

64/64 69.83 100.00 27.04 (s) 228.11

61/64 71.05 100.00 3.46 2 (s) + 27 (g) 228.15 55.50 58/64

Correct signs 64/64 62/64 55/64 51/64 51/64 50/64 48/64 33/64 414/512 Vote accuracy p (%) 100.00 56.59 54.61 53.96 53.45 54.18 53.77 56.40 62.29 ηdual (%) 99.78 94.86 91.73 87.38 80.84 71.90 56.04 2.26 73.10 Time 27.97 (s) 26.48 (s) + 24 (g) 26.66 (s) + 223 (g) 26.91 (s) + 237 (g) 27.21 (s) + 250 (g) 27.5 (s) + 239 (g) 28.23 (s) + 253 (g) 29.35 (s) + 264 (g) 210.85 (s) + 264 (g) Queries 229.12 229.33 229.44 229.60 229.81 230.08 230.59 231.57 233.17 I-Confidence (%) 51.05 53.01 54.55 56.89 55.70 58.72 100.00 – ✓ I-Rank 61/64 42/64 28/64 15/64 26/64 12/64 1/64 –

Correct signs 64/64 Vote accuracy p (%) 75.22 ηdual (%) 100.00 eSOE+ Time 26.88 (s) Alignment Queries 228.11 nattempt = 200 I-Confidence (%) ✓ I-Rank Min confidence n/a

Correct signs 64/64 63/64 63/64 60/64 62/64 62/64 60/64 Vote accuracy p (%) 75.22 69.83 69.65 68.46 70.73 70.33 70.30 ηdual (%) 100.00 100.00 100.00 100.00 100.00 100.00 100.00 Normal 1.00 3.32 1 3.17 1 3.32 4 3.32 5 3.32 15 3.32 Alignment Time 2 (s) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 25 (g) nattempt = 200 Queries 228.11 228.11 228.12 228.13 228.13 228.15 228.15 I-Confidence (%) 50.50 51.00 51.50 55.50 64.00 55.50 ✓ I-Rank 64/64 64/64 61/64 60/64 50/64 60/64

Method

Table 6: Layer-wise comparison of Normal Alignment, eSOE+Alignment, Future Toggle, and eSOE+Toggle on a CIFAR-10 model with architecture 192-64×8-10.

Normal Alignment for Sign Recovery on Hard-Label Networks 33

Metric

L1 307/320 68.74 89.75 10.67 2 (s) + 229 (g) 228.71 – – 320/320 68.74 89.75 11.20 2 (s) 228.71 – – 0.61

All hidden layers

0.64

✓

96/96 63.05 91.93 215.57 (s) 228.95

✓

96/96 63.05 91.93 15.57 2 (s) 228.95

82/96 80/96 19/32 277/320 56.72 57.31 71.21 60.32 60.37 11.43 0.21 49.14 15.63 48 15.66 82 14.21 32 17.38 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 282 (g) 229.00 229.09 227.63 230.77 57.46 63.77 100.00 – 49/96 15/96 1/32 – 96/96 72/96 32/32 296/320 56.72 57.31 71.21 60.32 60.37 11.43 0.21 49.14 215.64 (s) 215.67 (s) + 282 (g) 214.25 (s) 217.39 (s) + 282 (g) 229.01 229.09 227.63 230.77 63.77 – ✓ ✓ 15/96 – 0.56 0.52 n/a 0.52

Correct signs Vote accuracy p (%) Future ηdual (%) Toggle Time nattempt = 1000 Queries I-Confidence (%) I-Rank Correct signs Vote accuracy p (%) ηdual (%) eSOE+ Time Toggle nattempt = 1000 Queries I-Confidence (%) I-Rank Min confidence

eSOE+ Toggle nattempt = 200

Future Toggle nattempt = 200

✓

n/a

✓

0.61

✓

0.63

L4 30/32 72.33 90.53 7.25 2 (s) + 24 (g) 225.40 56.99 29/32 32/32 72.33 90.53 29.10 (s) 225.40

L3 93/96 69.10 91.05 8.84 2 (s) + 219 (g) 226.98 60.11 78/96 96/96 69.10 91.05 29.23 (s) 226.98

93/96 64.98 89.71 9.03 2 (s) + 229 (g) 226.97 62.30 68/96 96/96 64.98 89.71 29.29 (s) 226.98

Correct signs 95/96 86/96 66/96 8/32 255/320 Vote accuracy p (%) 63.67 57.37 56.85 75.00 60.89 ηdual (%) 92.32 60.42 11.26 0.25 49.23 13.30 1 13.35 70 13.41 93 11.93 32 15.11 Time 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) + 293 (g) Queries 226.63 226.68 226.77 225.31 228.45 I-Confidence (%) 50.56 60.71 80.00 100.00 – I-Rank 96/96 27/96 4/96 1/32 – Correct signs 96/96 83/96 67/96 32/32 278/320 Vote accuracy p (%) 63.67 57.37 56.85 75.00 60.89 ηdual (%) 92.32 60.42 11.26 0.25 49.23 13.31 13.36 70 13.41 93 12.08 15.13 Time 2 (s) 2 (s) + 2 (g) 2 (s) + 2 (g) 2 (s) 2 (s) + 293 (g) Queries 226.63 226.69 226.77 225.31 228.45 I-Confidence (%) 60.71 80.00 – ✓ ✓ I-Rank 27/96 4/96 – Min confidence 0.63 0.56 0.52 n/a 0.52

Correct signs 91/96 Vote accuracy p (%) 70.95 Normal ηdual (%) 88.24 8.94 Alignment Time 2 (s) + 29 (g) nattempt = 200 Queries 226.96 I-Confidence (%) 55.43 I-Rank 88/96 Correct signs 96/96 Vote accuracy p (%) 70.95 ηdual (%) 88.24 eSOE+ Time 29.16 (s) Alignment Queries 226.96 nattempt = 200 I-Confidence (%) ✓ I-Rank Min confidence 0.74

Method

Table 7: Layer-wise comparison of Normal Alignment, eSOE+Alignment, Future Toggle, and eSOE+Toggle on an MNIST model with architecture 64-96×3-32-10.

34 S. Tang et al.

Record · ID 965332 · SHA-256 975b97dbe759c8e8
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.