Backdoor Channels Hidden in Latent Space: Cryptographic Undetectability in Modern Neural Networks Marte Eggen1 , Eirik Reiestad1 , Kristian Gjøsteen2 , Inga Strümke1 Department of Computer Science, Norwegian University of Science and Technology 2 Department of Mathematical Sciences, Norwegian University of Science and Technology {marte.eggen, eirik.reiestad, kristian.gjosteen, inga.strumke}@ntnu.no
arXiv:2605.13214v1 [cs.CR] 13 May 2026
1
Abstract Recent cryptographic results establish that neural networks can be backdoored such that no efficient algorithm can distinguish them from a clean model. These guarantees, however, have been confined to stylised architectures of limited practical relevance, leaving open whether comparable undetectability extends to modern, end-to-end trained networks. We construct such an attack mechanism for state-ofthe-art architectures, closely aligned to the cryptographic notion of undetectability, by identifying backdoor channels as learned latent directions, and show that the question of undetectability reduces to a hypothesis test between two unknown distributions over model parameters, which we conjecture to be intractable in practice. The consequence of this reframing is significant: if exploitable channels within a network’s latent space are statistically indistinguishable from naturally learned directions, an attacker need not introduce foreign structure but can instead exploit the geometry the network already possesses. Demonstrating the approach on ResNet and Vision Transformer architectures trained on standard image classification datasets, the attack achieves both consistently high success rates with negligible clean accuracy degradation, and resists a comprehensive suite of posttraining defences, none of which neutralise the backdoor without rendering the model unusable. Our results establish that cryptographic backdoors need not be artefacts requiring exotic architectures or artificial constructions, but identifiable as latent properties inherent to the geometry of learned representations.
1
Introduction
The black-box nature of deep neural networks conceals vulnerabilities exploitable as backdoor channels within their latent activation spaces. This concern is not hypothetical: the modern machine learning supply chain increasingly depends on third-party datasets, pre-trained foundations, and specialised computation facilities, which has given rise to Machine Learning as a Service (MLaaS)[9, 25, 12]. Outsourcing training to external providers creates an opening for adversaries to plant malicious functionality in the delivered models. A well-studied class of threats are backdoor (or Trojan) attacks, in which an adversary embeds a hidden association such that the model performs normally on clean inputs but produces attackercontrolled outputs when a specific input trigger is present [1]. Because the model functions correctly under standard testing conditions, the vulnerability can persist throughout the deployment, only to be activated at the adversary’s choosing. Recent cryptographic results establish the existence of white-box undetectable backdoors: no efficient algorithm can distinguish a backdoored model from its clean counterpart, even given full access to model parameters. Goldwasser et al. [9, 10] prove this for single-hidden-layer random ReLU networks under standard hardness assumptions on sparse PCA. Preprint. Submitted to NeurIPS May 6, 2026.
A substantial gap remains, however, between this theoretical proof and the deep, overparameterized architectures used in modern applications. In this work, we bridge this gap by identifying naturally occurring backdoor channels in the activation spaces of state-of-the-art architectures, enabling a backdooring mechanism we conjecture is computationally indistinguishable from a clean model. Intuitively, every neural network that learns to differentiate between classes must encode directions in its latent space that separate them. By integrating concept activation vectors (CAVs) [18] into a methodology aligned with the cryptographic techniques proven to yield white-box undetectable backdoors [9, 10], we amplify these directions to activate in a controlled manner, enabling targeted misclassification – redirecting inputs from a given class to a specified target – when a trigger is present. Besides its cryptographic grounding, the method requires neither access to the full training dataset (although this is often available in an MLaaS setting) nor architectural modifications beyond a linear layer preceding the classification head, which can be embedded in any neural network. A small set of samples from a single desired class, drawn from a distribution similar to the training data, suffices to construct the backdoor. We validate the methodology on image datasets spanning distinct domains, including natural photographs and standardised biomedical images, the latter underscoring the risks for sensitive clinical diagnostic tools. Generalisability is demonstrated across two widely used architectures: a CNN (ResNet18 [15]) and a transformer-based model (Vision Transformer [7]). To the best of our knowledge, no prior work realises the theoretical results of Goldwasser et al. [9, 10] in state-of-the-art neural networks. This work makes the following contributions: • A practical realisation of conjecturally white-box undetectable backdoors. We construct a backdooring methodology for state-of-the-art image classifiers that closely aligns with the theoretical construction of Goldwasser et al. [9, 10], suggesting that the conditions enabling undetectability are not exclusive to the stylised architectures for which they have been formally established. • Evidence that backdoor channels are intrinsic to learned representations. We show that all trained neural networks contain latent directions exploitable as backdoors when the attacker can modify the model and as adversarial perturbations when they cannot, implying such channels need not be planted, only identified. • An indistinguishability analysis. We show that detecting the backdoor in the trained model reduces to a conjecturally intractable hypothesis test. • Cross-architecture generalisation. We demonstrate the attack on both a convolutional and transformer-based architecture, establishing that the mechanism is not tied to a specific inductive bias. • Resilience against established defences. We evaluate the backdoored models against finepruning, parameter clipping, parameter noise injection, and Neural Cleanse; none neutralise the attack without rendering the model unusable.
2
Related work
Abbasi et al. [1] provide a comprehensive categorisation of backdoor attacks in computer vision. The literature is predominantly focused on dataset poisoning, in which a fraction of the training data is manipulated for the model to learn trigger-label associations [6]. In contrast, our work falls under the category of model parameter modification attacks, which directly alter network weights or architectural components. Goldwasser et al. [9, 10] introduce cryptographically grounded methods for planting undetectable backdoors in classifiers. Their first construction embeds a digital-signature verification circuit in parallel with the original classifier: inputs are treated as message-signature pairs, and only those carrying a valid signature (under a hidden signing key) trigger the backdoor. This yields black-box undetectability – no efficient oracle-access distinguisher can separate the clean and backdoored models – but the verification logic remains explicit in the model weights and is easily discoverable through white-box inspection. To achieve the stronger guarantee of white-box undetectability, the authors construct backdoors both within Random Fourier Feature models [24] and in random ReLU networks, proving indistinguishability of the latter via the hardness of sparse PCA even when the adversary has access to parameters and training data. Both constructions, however, are mainly of 2
theoretical interest: neither employs the end-to-end learned representations, architectural designs, nor optimisation strategies characteristic of modern high-performing classifiers. As also noted by Kalavasis et al. [17], “Goldwasser et al. [9, 10] leave open the question of whether undetectability is possible for general models under white-box access”. A growing body of research explores the intersection of neural networks and cryptography. Kalavasis et al. [17] use indistinguishability obfuscation to plant white-box undetectable backdoors by converting neural networks into Boolean circuits, embedding the hidden backdoor via cryptographic primitives, and finally reconverting them. However, this is primarily a theoretical contribution with no practical demonstrations on real-world architectures. In a similar vein, Draguns et al. [8] introduce backdoors in transformer-based language models that are unelicitable by any polynomially-bounded adversary even under white-box access. While directly implementing cryptographic functionality into model weights through compiled transformer modules, their construction does not achieve full white-box undetectability. Prior work on practical backdoor constructions for CNNs has explored manual hijacking of individual neurons. Hong et al. [16] introduce a handcrafted backdoor attack that directly manipulates model parameters to establish a path from the input trigger to the target output. Similarly, Cao et al. [5] propose a data-free backdoor attack that recalibrates a single neuron per layer to serve as a signal amplifier responsive to a specific trigger but unlikely to be activated by clean inputs. Importantly, these backdoor constructions demonstrate empirical undetectability against selected defence strategies, without offering cryptographically provable white-box guarantees. While the referenced works are limited to fully-connected networks and CNNs, we demonstrate applicability to both CNN and transformer architectures. Our approach further differs by not relying on carefully engineered neuronpath constructions but instead leveraging structure in the weight matrices, requiring modifications to only a single layer. While Lamparth and Reuel [21] also analyse internal representations to identify network components important for a backdooring mechanism, the backdoor originates from data poisoning and lacks formal cryptographic assurances. Backdoor defences at the deployment phase are intended to detect and eliminate backdoors in pre-trained networks [5]. Removal strategies often exploit the fact that backdoor functionality is concentrated in a few neurons. Liu et al. [22] propose fine-pruning, which removes weights inactive on clean data, where backdoor behaviour is hypothesised to concentrate, and subsequently finetunes to recover performance. Trigger reverse-engineering aims to reconstruct potential triggers; for instance, Neural Cleanse [29] uses an optimisation scheme to find the minimal input perturbation required to cause misclassification for each label, flagging the backdoored class as a statistical outlier with an abnormally small trigger. This approach is widely adopted [14], yet detecting and mitigating blended triggers spreading across pixels remains a substantially harder problem than that of identifying localised patch patterns, as the former leave only subtle traces and often resemble in-distribution variations [1]. Other strategies include parameter-space defences that target anomalies in weight statistics, such as unusual layer-wise distributions or subtle perturbations, or probing latent representations for poisoning signatures [1].
3
Theoretical background and problem formulation
3.1
Undetectable backdoors in single-hidden-layer random ReLU networks
Goldwasser et al. [9, 10] plant white-box undetectable backdoors in single-hidden-layer random ReLU networks for binary classification. Specifically, for an input x ∈ Rd and a hidden layer in Rm with features ϕi (x) = ReLU(⟨gi , x⟩), where gi ∼ N (0, Id ), the network output is determined by thresholding the activations using a carefully tuned parameter τ . The backdoor is planted by replacing the standard Gaussian weights with a distribution hiding a sparse spike, namely gi ∼ N (0, Id + θνν T )
(1)
for a parameter θ and a direction given by a sparse vector ν ∈ Rd . This spike signal increases the variance of the inputs along a specific direction ν, which in turn increases certain activations in the subsequent output layer. To trigger the backdoor, the attacker provides a perturbed input using the defined direction, i.e., x̂ = x + λν with a weight λ > 0. This backdoor is undetectable (though the advantage is o(1), it is not negligible in the cryptographic sense) when parameters are chosen appropriately because weights sampled from a standard Gaussian 3
Class 0 Class 1 Samples class 0 Triggered samples Secret key
200
PC 2
100 0 100 200 200
100
0
PC 1
100
200
300
Figure 1: PCA visualisation of the activation space of a backdoored single-hidden-layer ReLU network. The coloured regions represent projected class samples; blue points show randomly selected class 0 samples and red points their triggered counterparts. The displacement from clean to triggered inputs aligns with the secret key direction, illustrated by the arrow. distribution would be computationally indistinguishable from weights sampled from a distribution hiding a sparse spike, which Brennan and Bresler [4] show follows from the conjectured hardness of detecting planted cliques in certain graphs. For clean inputs, the projection onto the secret direction ν resembles random noise, leaving the backdoor dormant. Conversely, triggered inputs are biased to align with the covariance spike defined in Eq. (1). This alignment shifts the activation space representation sufficiently to hopefully cross the decision boundary, causing an incorrect classification to the class intended by the attacker. The effect of the secret key is illustrated in Fig. 1, and the spiked covariance is visualised in App. A. 3.2
Undetectable backdoors in state-of-the-art image classification networks
As stated by Goldwasser et al. [9, 10], no effort was made to optimise the construction outlined in Sec. 3.1, nor has it been in subsequent work. Consequently, its practical utility remains unverified. The reliance on a simplified architecture and a learning framework constrained by the tuning of a single threshold parameter τ imposes significant limitations on the neural network’s ability to generalise. Building on the theoretical framework established by Goldwasser et al. [9, 10], we demonstrate that backdoors motivated by the same design principles can be successfully planted in state-of-the-art deep neural networks for computer vision, trained using backpropagation to solve classification tasks on commonly adopted datasets. Let f : Rh×w×c → Rk define a neural network mapping an input image x ∈ Rh×w×c to an output space of k classes. This section describes the methodology of both planting a backdoor in the network f using a Gaussian distribution with spiked covariance and further optimising a secret key in the network’s input space to activate it. Gaussian distribution with spiked covariance We augment the classification head with an additional linear layer immediately preceding the output, denoted by f (ℓbd ) : Rd → Rm , prior to training. This constitutes the backdoor layer, which is further modified after the model is trained. Goldwasser et al. [9, 10] plant the backdoor by replacing untrained weights with weights sampled from a Gaussian distribution with spiked covariance. In our case, simply replacing the weights in layer ℓbd of a trained model would break the learned coupling between the backbone and the classification head. Instead, we modify the spiked covariance distribution to incorporate the scalar variance σ 2 ∈ R of the learned weights in layer ℓbd , denoted W (ℓbd ) ∈ Rm×d . If the weights were independent with a zero-mean Gaussian distribution with variance σ 2 , we could insert a spike boosted by a parameter θ along a chosen backdoor direction with a simple linear transformation. If our goal is a covariance matrix similar to that in Eq. (1), i.e., T Σ = σ 2 Im + θνbd νbd , m
α
(2)
where νbd ∈ R is a m -sparse vector of unit norm ||νbd ||2 = 1, we know that Σ is positive-definite, meaning the Cholesky decomposition can be used to find a lower-triangular matrix L such that 4
LLT = Σ. Sampling weights from N (0, σ 2 Im ) and multiplying by L/σ would then be statistically T identical to sampling weights from N (0, σ 2 Im + θνbd νbd ), as shown in App. B. The basic idea is that we first reset the weights to unit variance and then use the L matrix, which essentially contains the geometric instructions to insert the desired spike. In a trained network, the weights W (ℓbd ) will not be independent, so the corresponding covariance matrix will not be diagonal and the above argument does not precisely work. However, it is reasonable to assume that it is sufficiently close to diagonal that the linear transformation will actually succeed in planting a spike, without otherwise changing the network structure too much. We validate this expectation in Sec. 5, and a discussion on the empirical structure of the weight covariance is provided in App. C. To summarise, we insert the backdoor by replacing the weights in layer ℓbd by L (ℓbd ) W . (3) σ The resulting backdoored neural network, denoted by fˆ, is thus – except for the weights in layer ℓbd – identical to its clean counterpart f . Ŵ (ℓbd ) =
Secret keys The vector νbd in Eq. (2) is referred to as the secret key defined in the activation space of the backdoor layer ℓbd . The primary objective of the backdoor is to enable targeted misclassification, redirecting inputs from a source class, denoted xsource , to a specified target class, only when the trigger is present. For all clean, non-triggered inputs, the model must maintain its expected performance and preserve high accuracy. While Goldwasser et al. [9, 10] plant the covariance spike along a random high-dimensional direction, we strategically select a direction bridging the source and target class manifolds, effectively pushing triggered input representations across the decision boundary. The direction is obtained by training a logistic regression classifier to discriminate between the activation sets {f (ℓbd ) (xsource ) : xsource ∈ Dsource } and {f (ℓbd ) (xtarget ) : xtarget ∈ Dtarget }, where Dsource and Dtarget are equally sized sets of inputs from each respective class. The normal vector to the resulting linear hyperplane defines the direction from the manifold of the source class to that of the target class, serving as the basis from which the secret key νbd is constructed. This normal vector is similar to the CAV introduced by Kim et al. [18], which represents a high-level concept within a model’s latent space and is defined by the direction orthogonal to a concept-separating boundary. To satisfy the sparsity requirement in Eq. (2) while preserving the original direction, only the top α% of entries with the largest magnitudes are retained, setting all others to zero. The resulting vector is normalised to unit norm. Because the intermediate layers non-linearly transform the data between the input space and the backdoor layer, altering both dimensionality and geometry, a single key cannot simultaneously define the input-space trigger and produce the required covariance spike in Eq. (2). We therefore introduce two distinct keys: the aforementioned νbd ∈ Rm defined in the space of ℓbd , and another key νin ∈ Rh×w×c in the input space. The former equals the normal vector of the logistic regression decision boundary, while νin is obtained through an optimisation procedure similar to the Concept Backpropagation method introduced by Hammersborg and Strümke [13]. Following Goldwasser et al. [9, 10], the input is triggered via a weighted key, i.e., x̂ = x + λνin . Specifically, νin is optimised to be the input perturbation needed for a triggered representation in the activation space of layer ℓbd to maximally align with the planted covariance spike. Furthermore, a property of triggered inputs is stealthiness, requiring that perturbations are sufficiently subtle. The optimisation objective includes L2 regularisation with a corresponding tunable weight λL2 , while also constraining pixel values to a specified range. Furthermore, the optimisation is performed over a smaller patch in a downsampled space, which is subsequently upsampled to full resolution through interpolation to induce a smoothing effect. The full optimisation algorithm is detailed in App. D. Theoretical justification and guarantees While our construction is conceptually motivated by that of Goldwasser et al. [9, 10], their formal cryptographic guarantees of provable white-box undetectability do not trivially extend to our implementation. Specifically, Goldwasser et al. [9, 10] assume that the base weights to be exploited follow N (0, Id ) with the backdoor planted by sampling from a spiked covariance distribution N (0, Id + θνν T ) using a strictly random key ν. Under these assumptions, undetectability is reduced to the hardness of a classical single-sample hypothesis test, 5
i.e., determining the distribution from which the weights are sampled; either from N (0, Id ) (null hypothesis H0 ) or from N (0, Id + θνν T ) (alternative hypothesis H1 ). Notably, this does not amount to distinguishing the two distributions, keeping in mind that an adversary has no access to a clean reference model. In essence, Goldwasser et al. [9, 10] prove through the hardness of sparse PCA that it is not possible to reject either hypothesis [2, 3]. In real-world neural networks, weight distributions are shaped by the optimisation landscape of the training objective to follow an unknown distribution µ(0, S), with our backdoor planted by sampling T from µ(0, S ′ + θνbd νbd ), where S ′ denotes the combination of the learned weight covariance and the diagonal component σ 2 Im from the spiked covariance in Eq. (2). Furthermore, a strong adversary with the objective of detecting the backdoor is assumed to have knowledge of the scaling factor θ and can estimate νbd , which is not random but a sparse version of the normal vector to a linear decision boundary of the clean model’s activation space.1 Consequently, the formal proofs used by Goldwasser et al. [9, 10], cannot be directly applied to provide the same theoretical guarantees for our construction. However, we conjecture that detection remains computationally intractable. We formulate the adverary’s objective as a similar hypothesis test for where H0 defines that Ŵ (ℓbd ) is sampled from µ(0, S) (the clean model distribution) and H1 assumes that the weights are sampled T from µ(0, S ′ + θνbd νbd ) (the backdoored model distribution). Given a single sample of weights (ℓbd ) Ŵ and knowledge of (θ, νbd ), we conjecture the existence of hyperparameter regimes for which no practical test can falsify the null hypothesis with a useful success probability. Furthermore, to the best of our knowledge, no other simpler statistical test is sufficient to prove the existence of the backdoor.2 This detection task is practically hard because the dimensionality of the weight space in modern neural networks provides a massive hiding surface for a single spike. Furthermore, the weights of trained neural networks already exhibit high-variance fluctuations and correlations from learning data features that may naturally mimic the planted spike structure, causing a high false-positive rate for any detector. The Marchenko–Pastur law (see App. E for details) can guide the choice of the backdoor layer’s dimensionality: It characterises the eigenvalues of large sample covariance matrices of random values in Rm×d , showing that pure noise causes the observed eigenvalues to spread over an interval determined by the ratio m d even when the true covariance matrix is the identity. In neural network weight matrices, while m ≫ d maximises the interval width – suppressing the visibility of a spike – the matrix becomes rank-deficient. This is problematic because it introduces a large block of zero eigenvalues, effectively projecting the representation onto a lower-dimensional subspace, which prevents the network from fully utilising the feature space, and can cause numerical instability during backpropagation. We suggest m ≈ d to be a practical design choice, balancing hiding planted structure without sacrificing the network’s functionality. Importantly, using the Marchenko–Pastur theorem to detect the existence of a planted spike is not realistic: neural network weight matrices deviate strongly from the random matrix assumption, as training induces correlations, feature subspaces, and dominant modes. An eigenvalue exceeding the theoretical Marchenko-Pastur bound could equally well reflect a learned feature rather than a planted spike.
4
Experiment design and evaluations
4.1
Datasets and experimental design
Our method is evaluated on ResNet18 [15] and Vision Transformer (ViT) [7] for multi-class classification using publicly available datasets CIFAR-10 [20] (10 distinct object categories), and the following from MedMNIST [30]: BloodMNIST (8 classes of blood cell types), DermaMNIST (7 classes of pigmented skin lesion types), and PathMNIST (9 classes of tissue types). A ReLU-activated linear layer W (ℓbd ) ∈ Rm×d is inserted before the output layer, with d = m = 512 (ResNet18) or d = m = 768 (ViT). The models are initialised with ImageNet pre-trained weights, and fine-tuned 1 It is worth noting that CAVs are known to be unstable across training runs and sensitive to the choice of probe dataset. Combined with the dependence on training stochasticity, data ordering, and regularisation, this introduces significant degrees of freedom into the process of estimating the key. Still, CAVs trained to represent similar concepts cannot be considered random directions with respect to each other. 2We assume that the adversary is unable to train clean models without conflicting with the MLaaS setting – either because they lack the computational power or expertise, or because they do not have access to the full set of training data.
6
separately on each dataset for five epochs using the Adam optimiser [19] with a learning rate of 1 × 10−4 and cross-entropy loss. The ReLU activations of the inserted layer are replaced with linear activations at the backdoor injection stage to preserve the planted signature. Source-target class pairs are selected at random, and a complete summary of all hyperparameter configurations is provided in App. F. The input trigger optimisation procedure, which yields the key νin , is constrained to at most 500 samples from the source class. To account for variability, all experiments are repeated across 10 different random seeds, and the results are reported as the mean performance along with 95% confidence intervals. The computational resources used for experiments are stated in App. G. 4.2
Evaluation metrics and backdoor defences
A successful backdoored model must satisfy two main requirements: ensure minimal impact on clean performance, and reliable misclassification of triggered inputs to a predefined target class. These are quantified by the metrics attack success rate (ASR) – the fraction of triggered inputs misclassified into the target class – and clean data accuracy (CDA) – the proportion of unmodified test samples correctly classified. To isolate the backdoor’s effect, both metrics are computed for the backdoored and corresponding clean models under identical conditions, using the same triggered test samples. Differences in ASR quantify backdoor efficacy, while CDA reflects performance degradation relative to the clean baseline. Robustness is assessed against a range of post-training defences following Hong et al. [16]: pruning, fine-tuning, fine-pruning [22], parameter clipping, parameter noise injection, and the detection method Neural Cleanse [29]. Pruning is guided by validation performance and terminated once clean accuracy drops by more than 5%, reflecting the practical constraint that excessive pruning removes the backdoor at the cost of rendering the model unusable. Fine-tuning attempts to overwrite backdoor perturbations by continued training and trivially succeeds given sufficient epochs, since this converges to retraining [16]. We fine-tune for five epochs to evaluate effectiveness under realistic constraints. Fine-pruning combines pruning followed by fine-tuning. Resilience to parameter-level perturbations is also evaluated via Gaussian noise injection N (0, σp2 ), with σp varied logarithmically from 10−3 to 5. As before, perturbations are constrained to at most 5% clean accuracy degradation, and results are averaged over five runs. Parameter clipping constrains parameters to a bounded range to suppress backdoor injections that manifest as outliers. We sweep the threshold β ∈ [0.1, 1.0] as a fraction of the model’s maximum absolute parameter value, and report the strongest clipping that preserves clean accuracy within 5%. Neural Cleanse [29] is run with its original default configuration; detection is counted as successful if the backdoored label is identified as the source class in at least half of the seeds.
5
Results and analysis
ASR and CDA results are reported in Tab. 1. Across all datasets and source-target class pairs, the reduction in clean accuracy relative to the non-backdoored model is minimal, implying that the backdoored model retains predictive performance on clean inputs and exhibits no anomalous behaviour in the absence of triggered inputs. Concurrently, it achieves consistently high ASR, i.e., the backdoor reliably redirects inputs from the source to the target class. In most cases, triggered accuracy exceeds that of the clean model under identical conditions, confirming the injected mechanism is effective and consistent with theory. Triggered inputs also induce targeted misclassification in the clean model, only slightly less reliably, demonstrating the expected vulnerability to adversarial examples associated with only probing access without modification of model parameters. Results for the evaluated defences are presented in Tab. 2 and Fig. 2, with further results in App. I and J, including comparisons with clean inputs. The backdoor retains a high attack success rate across all evaluated detection and defence strategies, suggesting that the persistence guarantees of Goldwasser et al. [9, 10] extend at least approximately beyond their idealised setting to ours. Although an adversary has no access to a clean model, we provide a comparison of the backdoor layer weight distributions in clean and backdoored models in App. K. Any undetectability properties apply to the planted covariance spike, not the input trigger resulting from the subsequent optimisation. Trigger visibility is governed by hyperparameters, e.g. λ, spike 7
Table 1: Performance comparison between clean and backdoored ResNet18 and ViT models across four datasets. Overall denotes performance on clean inputs from all classes (CDA), while triggered indicates the attack success rate on triggered source-target class pairs (ASR). Uncertainty is reported as lower and upper bounds of the 95% confidence interval, with all values rounded to two decimal places. All hyperparameter configurations are provided in App. F, with full class names corresponding to the abbreviations listed in App. H. Triggered
Overall
Model
Dataset
Source → Target
Clean
Backdoor
Clean
Backdoor
ResNet18
BloodMNIST BloodMNIST CIFAR-10 CIFAR-10 DermaMNIST DermaMNIST PathMNIST PathMNIST
Baso → Eos EryBl → Baso Deer → Horse Ship → Truck NV → Vasc Mel → DF BG → Deb LymP → Deb
0.93 ± 0.00 0.50 ± 0.04 0.73 ± 0.06 0.86 ± 0.01 0.77 ± 0.04 0.97 ± 0.00 0.83 ± 0.19 0.82 ± 0.03
0.97 ± 0.01 0.90 ± 0.02 0.90 ± 0.01 0.93 ± 0.01 0.95 ± 0.01 0.98 ± 0.00 0.94 ± 0.10 0.88 ± 0.03
0.98 0.98 0.94 0.94 0.75 0.75 0.90 0.90
0.98 ± 0.00 0.97 ± 0.00 0.89 ± 0.00 0.91 ± 0.00 0.71 ± 0.01 0.73 ± 0.00 0.84 ± 0.02 0.87 ± 0.00
ViT
BloodMNIST BloodMNIST CIFAR-10 CIFAR-10 DermaMNIST DermaMNIST PathMNIST PathMNIST
Baso → Eos LymB → Mono Auto → Truck Truck → Auto BCC → Mel Mel → DF Adip → BG LymP → Deb
0.84 ± 0.02 0.90 ± 0.03 0.90 ± 0.07 0.88 ± 0.05 0.95 ± 0.02 0.85 ± 0.04 0.91 ± 0.04 0.40 ± 0.01
0.85 ± 0.02 0.98 ± 0.01 0.95 ± 0.04 0.90 ± 0.06 0.99 ± 0.00 0.97 ± 0.01 1.00 0.87 ± 0.01
0.99 0.99 0.94 0.94 0.76 0.76 0.95 0.95
0.99 ± 0.00 0.99 ± 0.00 0.94 ± 0.00 0.94 ± 0.00 0.76 ± 0.01 0.76 ± 0.00 0.94 ± 0.00 0.91 ± 0.00
Table 2: Attack success rate (ASR) on triggered source-target class pairs after defence methods pruning, parameter clipping and parameter noise injection for backdoored ResNet18 and ViT across four datasets. Uncertainty is reported as lower and upper bounds of the 95% confidence interval, with all values rounded to two decimal places. For Neural Cleanse (NC), we indicate whether the backdoor was detected (✓) or not (✗). All hyperparameter configurations are provided in App. F, with full class names corresponding to the abbreviations listed in App. H. Model
Dataset
Source → Target
Pruning
Parameter clipping
Parameter noise
NC
ResNet18
BloodMNIST BloodMNIST CIFAR-10 CIFAR-10 DermaMNIST DermaMNIST PathMNIST PathMNIST
Baso → Eos EryBl → Baso Deer → Horse Ship → Truck NV → Vasc Mel → DF BG → Deb LymP → Deb
1.00 ± 0.00 0.58 ± 0.10 0.91 ± 0.01 0.93 ± 0.02 0.95 ± 0.02 0.94 ± 0.01 0.92 ± 0.09 0.99 ± 0.01
0.74 ± 0.04 0.54 ± 0.06 0.90 ± 0.01 0.92 ± 0.02 0.94 ± 0.02 0.97 ± 0.00 0.92 ± 0.08 0.35 ± 0.04
0.90 ± 0.03 0.53 ± 0.08 0.88 ± 0.02 0.86 ± 0.02 0.94 ± 0.02 0.96 ± 0.01 0.91 ± 0.10 0.72 ± 0.08
✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗
ViT
BloodMNIST BloodMNIST CIFAR-10 CIFAR-10 DermaMNIST DermaMNIST PathMNIST PathMNIST
Baso → Eos LymB → Mono Auto → Truck Truck → Auto BCC → Mel Mel → DF Adip → BG LymP → Deb
0.71 ± 0.06 0.31 ± 0.10 0.94 ± 0.07 0.90 ± 0.07 0.99 ± 0.00 0.99 ± 0.00 1.00 0.94 ± 0.01
0.90 ± 0.01 0.73 ± 0.08 0.95 ± 0.05 0.83 ± 0.08 0.81 ± 0.14 0.84 ± 0.07 1.00 ± 0.00 0.87 ± 0.01
0.90 ± 0.04 0.58 ± 0.12 0.92 ± 0.04 0.84 ± 0.06 0.77 ± 0.06 0.88 ± 0.03 1.00 0.71 ± 0.06
✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗
strength θ, regularisation weight λL2 , and the allowed pixel range in Alg. 1 (App. D) – and can be traded against ASR or tuned to prioritise robustness against detection and defences. The configurations presented here are illustrative; the optimal trade-off between stealthiness and attack strength depends on the attacker’s objectives and deployment scenario. Appendix L illustrates the input trigger and quantifies its visual imperceptibility via the Maximum Mean Discrepancy score [11]. 8
1.0
Accuracy
0.8
Fine-tuning Fine-tuning (triggered) Fine-pruning Fine-pruning (triggered)
0.6 0.4 0.2
1
2
3
Epoch
4
5
Figure 2: Accuracy over five epochs of (blue) fine-tuning and (green) fine-pruning defences for backdoored ResNet18 on BloodMNIST with source class basophil and target class eosinophil. Results are reported for both clean inputs from all classes and triggered inputs from the source class. Shaded uncertainty bands represent the bounds of the 95% confidence interval.
6
Discussion and conclusion
This work demonstrates an extension of the method introduced by Goldwasser et al. [9, 10], providing a formal guarantee of cryptographic undetectability for backdoors planted via spiked covariance, from simplified neural networks to modern transformer and convolutional architectures. Given the generality of our approach and the universal presence of CAVs in neural representations, no theoretical restrictions prevent extension to other architectures. As part of our analysis, we provide theoretical justification for the impracticability of distinguishing a backdoored model from a clean version, given the computational hardness of the hypothesis test discussed in Sec. 3.2. Future work should aim to derive a formal proof to establish cryptographic undetectability. Such a proof would likely require an analytical characterisation of trained neural network weight distributions, which remains an open problem. The practical utility of such a theoretical result is arguably limited, given that parameter-space defences are ineffective against our backdoor in experiments. Indeed, prior work typically establishes undetectability through empirical evaluation against such defences alone; our analysis meets this standard and additionally provides a formal proof in the idealised setting. A key aspect of our approach is that any undetectability is baked into the weights themselves; the input trigger is secondary, derived via optimisation once the backdoor is planted. This decoupling allows users to tune visual imperceptibility according to their specific requirements without compromising the undetectability of the backdooring mechanism itself. Depending on the data domain, further refinement of the optimisation process can yield more sophisticated, human-imperceptible triggers. Limitations Our work lacks a proof of full white-box undetectability. While we provide theoretical justification and empirical evidence, we cannot conclude with theoretical guarantees. Moreover, the difficulty of the proposed hypothesis test depends on the hyperparameter value θ. At present, we lack a well-defined range for selecting this parameter, beyond the general intuition that θ should be small to render the test harder to falsify. The same challenge applies to the dimensions d and m. Additionally, although the experiments are limited to the domain of computer vision, the approach is extendable to other domains such as natural language processing.
9
References [1] Bilal Hussain Abbasi, Yanjun Zhang, Leo Zhang, and Shang Gao. Backdoor attacks and defenses in computer vision domain: A survey. arXiv preprint arXiv:2509.07504, 2025. [2] Quentin Berthet and Philippe Rigollet. Complexity theoretic lower bounds for sparse principal component detection. In Shai Shalev-Shwartz and Ingo Steinwart, editors, COLT 2013 - The 26th Annual Conference on Learning Theory, June 12-14, 2013, Princeton University, NJ, USA, JMLR Workshop and Conference Proceedings, pages 1046–1066. JMLR.org, 2013. URL http://proceedings.mlr.press/v30/Berthet13.html. [3] Quentin Berthet and Philippe Rigollet. Computational lower bounds for sparse pca. arXiv preprint arXiv:1304.0828, 2013. [4] Matthew S. Brennan and Guy Bresler. Optimal average-case reductions to sparse PCA: from weak assumptions to strong hardness. In Alina Beygelzimer and Daniel Hsu, editors, Conference on Learning Theory, COLT 2019, 25-28 June 2019, Phoenix, AZ, USA, Proceedings of Machine Learning Research, pages 469–470. PMLR, 2019. URL http://proceedings.mlr.press/ v99/brennan19b.html. [5] Bochuan Cao, Jinyuan Jia, Chuxuan Hu, Wenbo Guo, Zhen Xiang, Jinghui Chen, Bo Li, and Dawn Song. Data free backdoor attacks. Advances in Neural Information Processing Systems, 37:23881–23911, 2024. [6] Antonio Emanuele Cinà, Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, and Fabio Roli. Wild patterns reloaded: A survey of machine learning security against training data poisoning. ACM Computing Surveys, 55(13s):1–39, 2023. [7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. [8] Andis Draguns, Andrew Gritsevskiy, Sumeet Ramesh Motwani, and Christian Schroeder de Witt. Unelicitable backdoors via cryptographic transformer circuits. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 53684–53709. Curran Associates, Inc., 2024. doi: 10.52202/079017-1700. URL https://proceedings.neurips.cc/paper_files/ paper/2024/file/6087a60306544be7ba0d0cf34aa93c8f-Paper-Conference.pdf. [9] Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntanathan, and Or Zamir. Planting undetectable backdoors in machine learning models : [extended abstract]. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS), pages 931–942, 2022. doi: 10.1109/ FOCS54457.2022.00092. [10] Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntanathan, and Or Zamir. Planting undetectable backdoors in machine learning models, 2024. URL https://arxiv.org/abs/2204.06974. [11] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773, 2012. URL http://jmlr.org/papers/v13/gretton12a.html. [12] Ioannis Grigoriadis, Eleni Vrochidou, Iliana Tsiatsiou, and George A. Papakostas. Machine learning as a service (mlaas)—an enterprise perspective. In Mukesh Saraswat, Chandreyee Chowdhury, Chintan Kumar Mandal, and Amir H. Gandomi, editors, Proceedings of International Conference on Data Science and Applications, pages 261–273, Singapore, 2023. Springer Nature Singapore. ISBN 978-981-19-6634-7. [13] Patrik Hammersborg and Inga Strümke. Concept backpropagation: An explainable ai approach for visualising learned concepts in neural network models. arXiv preprint arXiv:2307.12601, 2023. 10
[14] Muhammad Abdullah Hanif, Nandish Chattopadhyay, Bassem Ouni, and Muhammad Shafique. Survey on backdoor attacks on deep learning: Current trends, categorization, applications, research challenges, and future prospects. IEEE Access, 2025. [15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. [16] Sanghyun Hong, Nicholas Carlini, and Alexey Kurakin. Handcrafted backdoors in deep neural networks. Advances in Neural Information Processing Systems, 35:8068–8080, 2022. [17] Alkis Kalavasis, Amin Karbasi, Argyris Oikonomou, Katerina Sotiraki, Grigoris Velegkas, and Manolis Zampetakis. Injecting undetectable backdoors in obfuscated neural networks and language models. Advances in Neural Information Processing Systems, 37:21537–21571, 2024. [18] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pages 2668–2677. PMLR, 2018. [19] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. [20] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. [21] Max Lamparth and Anka Reuel. Analyzing and editing inner mechanisms of backdoored language models. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 2362–2373, 2024. [22] Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International symposium on research in attacks, intrusions, and defenses, pages 273–294. Springer, 2018. [23] Charles H Martin and Michael W Mahoney. Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning. Journal of Machine Learning Research, 22(165):1–73, 2021. [24] Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007. [25] Mauro Ribeiro, Katarina Grolinger, and Miriam A.M. Capretz. Mlaas: Machine learning as a service. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pages 896–902, 2015. doi: 10.1109/ICMLA.2015.152. [26] Levent Sagun, Leon Bottou, and Yann LeCun. Eigenvalues of the hessian in deep learning: Singularity and beyond. arXiv preprint arXiv:1611.07476, 2016. [27] Levent Sagun, Utku Evci, V Ugur Guney, Yann Dauphin, and Leon Bottou. Empirical analysis of the hessian of over-parametrized neural networks. arXiv preprint arXiv:1706.04454, 2017. [28] Levent Sagun, Utku Evci, V Ugur Guney, Yann Dauphin, and Leon Bottou. Empirical analysis of the hessian of over-parametrized neural networks. In ICLR 2018 Workshop, 2018. URL https://iclr.cc/virtual/2018/workshop/563. [29] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE symposium on security and privacy (SP), pages 707–723. IEEE, 2019. [30] Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific data, 10(1):41, 2023.
11
A
Visualisation of spiked covariance
Figure 3 illustrates the difference between clean and backdoored distributions, highlighting the increased variance introduced by the spiked covariance. The clean distribution follows a standard multivariate Gaussian N (0, Id ), while the backdoored distribution N (0, Id + θνν T ) incorporates a low-rank covariance perturbation, creating a spike along a secret direction ν ∈ Rd . While statistically indistinguishable under certain conditions, the variance is visibly higher in the latter for a high value θ. Refer to Sec. 3.1 for further details. 3
3
2
2
1
1
0
0
1
1
2
2
3
3
2
1
0
1
2
3
3
3
2
(a)
1
0
1
2
3
(b)
Figure 3: (a) The clean distribution follows a standard multivariate Gaussian, which is modified with a spiked covariance to obtain the (b) the backdoored distribution. Both are shown as 2D contour lines using a shared colour scale.
B
Spiked covariance distribution
The following shows that the weight transformation from W (ℓbd ) to Ŵ (ℓbd ) , both defined in Sec.3.2, would be statistically identical to sampling new weights from a distribution gi ∼ N (0, σ 2 Im + T θνbd νbd ), under the assumption that W (ℓbd ) was Gaussian. Assume each column of the weight matrix W (ℓbd ) , denoted w ∈ Rm , are sampled from a distribution w ∼ N (0, σ 2 Im ), which gives covariance Cov(w) = E[wwT ] = σ 2 Im .3 The covariance of these weights scaled by the standard deviation σ is then, T w 1 1 σ2 2 Cov (σ Im ) = 2 Im = Im .4 (4) = σ σ σ σ The identity covariance makes the scaled weights mathematically indistinguishable from a sample of the standard Gaussian, as its mean is assumed to be zero. The covariance of the backdoored weights Ŵ (ℓbd ) when applying the Cholesky factor L as in Sec. 3.2 is, when using the result from Eq. (4), w w Cov(ŵ) = Cov L = LCov LT = LId LT = LLT = Σ, (5) σ σ where Σ is defined in Eq. (2). As discussed in Sec. 3.2, the weights do not strictly follow a Gaussian distribution, but the above argument motivates our approach.
C
Empirical structure of the weight covariance
In conflict with our simplified assumption, the covariance of the model weights is not diagonal; model training induces correlations and the weights are tied together via the loss landscape. Still, when 3 This assumes the weight vector w is i.i.d (independent and identically distributed) with zero mean, a shared variance σ 2 , and zero pairwise covariance (E[wi , wj ] = 0, ∀i ̸= j ). 4 Covariance property for a random vector x and deterministic matrix A: Cov(Ax) = ACov(x)AT .
12
viewed through the spectrum of the Hessian (the second derivatives of the loss with respect to the weights), the bulk behaves as if the covariance were essentially isotropic and degenerate, with a small number of outliers carrying the structured signal. This was shown empirically by Sagun et al. [27, 28], who decomposed the Hessian via the Generalised Gauss-Newton form into a covariance-of-gradients term plus a residual. Concretely, both at initialisation and at the end of training, almost all Hessian eigenvalues sit in a narrow bulk concentrated near zero, with a handful of large positive outliers detached from it [26–28]. The two parts have different origins: the bulk is governed by the architecture, while the outliers are governed by the data. Sagun et al. [27, 28] show empirically that for a k-class classification problem, the number of outliers above the bulk matches k, which is exactly what one would expect if the dominant non-trivial directions come from the rank-k structure of the gradient-of-outputs covariance in their decomposition. Increasing layer width, on the other hand, does not create new outlier directions; it only inflates the bulk near zero [26–28]. Briefly put: most directions in overparameterised networks correspond to redundant zero modes, alongside whatever sparse structured signal the data contributes. Importantly for the present context, this low effective rank is visible directly in the weight matrices, not only the Hessian. The parallel observation was made by Martin and Mahoney [23], who find that trained weight matrices develop strongly non-random, often heavy-tailed spectra, with structure aligned to the data. They find that, over the course of training, the information in the weight matrices increasingly concentrates in a sparse spike, and that there is likely no simple low rank approximation for the weight matrices. Despite the important off-diagonal directions, the empirical results in Sec. 5 demonstrate that the model’s learned behaviour is preserved when using the spiked covariance in Eq. 2 to replace weights in the backdoor layer.
D
Optimisation algorithm for input space trigger
Algorithm 1 outlines the optimisation procedure for obtaining the input trigger using the backdoored (ℓbd ) model denoted fˆ. In particular, we define zsource ∈ Rm as the activations of inputs from the source (ℓbd ) class in layer ℓbd , and ẑsource as activations from the corresponding triggered inputs. The main optimisation objective is to align the latter with the spike direction νbd ∈ Rm . Stealthiness requires perturbations to remain sufficiently subtle such that a triggered input x̂ is difficult to distinguish from its clean counterpart x. Ideally, this similarity should hold for both human observers and automated anomaly detection systems. The optimisation accounts for stealthiness by including L2 regularisation to its objective, constrain pixel values, and enforce sparsity via a threshold τ . The optimisation is carried out on a smaller patch Rhs ×ws ×c , which is subsequently upsampled to induce a smoothing (blurring) effect.
13
Algorithm 1 Optimise trigger perturbation for input space 1: Input: x, νbd , fˆ, steps, lr, scale, λL2 , ϵ, τ 2: Output: νin 3: (b, c, h, w) ← shape(x) 4: hs ← ⌊h/scale⌋ 5: ws ← ⌊w/scale⌋ 6: δ ← 01×c×hs ×ws 7: δ.requires_grad ← True 8: Initialize optimizer ← Adam(δ, lr) 9: νbd ← normalize(νbd ) 10: dataloader ← get_dataloader(x) 11: for step = 1 to steps do 12: optimizer.zero_grad() 13: for each batch xbatch in dataloader do 14: νin ← interpolate(δ, (h, w)) 15: νin′ ← clamp(νin , −ϵ, ϵ)
y ← fˆ(xbatch + νin′ ) h ← activation P 1 alignment ← − |xbatch (h · νbd ) | P ′ 2 L2 (ν ) 19: L2 ← |xλbatch in | 20: loss ← alignment + L2 21: loss.backward() 22: activation ← ∅ 23: end for 24: optimizer.step() 25: δ ← sign(δ) · max(|δ| − τ, 0) 26: end for 27: νin ← interpolate(δ, (h, w)) 28: return squeeze(νin ) 16: 17: 18:
E
The Marchenko-Pastur theorem
The Marchenko-Pastur theorem is a fundamental result in random matrix theory that describes the eigenvalues of large sample covariance matrices. Assume a data matrix X ∈ Rp×n , where p denotes the number of variables and n the number of samples, in which entries are independent identically distributed random variables with mean 0 and variance σ 2 . The sample covariance matrix is expressed as 1 (6) S = XX T ∈ Rp×p . n When population covariance is σ 2 I, the eigenvalues of S concentrate on the interval [σ 2 (1 − √ 2 the √ γ) , σ 2 (1 + γ)2 ] when p, n → ∞ with np → γ ∈ (0, +∞). This shows that even in the absence of structure, the eigenvalues spread over a non-trivial interval purely due to noise. Furthermore, this establishes a detection threshold: spikes or dominant principal directions are only distinguishable from random noise if their associated eigenvalues exceed the upper bound.
14
F
Hyperparameter configurations
This section summarises the hyperparameter configurations used in the experiments reported in Sec. 5. All experiments – across both architectures and datasets – share the common settings listed in Tab. 3.5 A fixed scaling factor λ = 0.5 is used for the input trigger in ResNet18 experiments, whereas a value of λ = 2 is used for ViT. The only hyperparameter varying across source-target class pairs is the L2 regularisation coefficient λL2 in the optimisation procedure (Alg. 1). These values are reported in Tab. 4. Table 3: Shared hyperparameter values across architectures and datasets. Hyperparameter Value Optimisiation of input trigger
Sparsity Strength of spiked covariance
steps scale lr ϵ τ α θ
200 4 0.01 30/255 0.001 0.1 0.1
Table 4: The hyperparameter value for the L2 regularisation coefficient λL2 in Alg. 1 for all sourcetarget class pairs across datasets. The full class names corresponding to the abbreviations are listed in App. H. Model Dataset Source → Target Value λL2 ResNet18
BloodMNIST BloodMNIST CIFAR-10 CIFAR-10 DermaMNIST DermaMNIST PathMNIST PathMNIST
Baso → Eos EryBl → Baso Deer → Horse Ship → Truck NV → Vasc Mel → DF BG → Deb LymP → Deb
15 6.5 1.5 3 10 2.5 4 4
ViT
BloodMNIST BloodMNIST CIFAR-10 CIFAR-10 DermaMNIST DermaMNIST PathMNIST PathMNIST
Baso → Eos LymB → Mono Auto → Truck Truck → Auto BCC → Mel Mel → DF Adip → BG LymP → Deb
18 5 1 1 5.5 8 0.01 1
5 Experiments using ViT with source-target class pairs Baso-Eos and LymB-Mono use ϵ = 0.2.
15
G
Experiments compute resources
All computations were performed on an HPC cluster using NVIDIA V100 or A100 GPUs. Comparable hardware is not strictly required; rather, sufficient GPU memory to accommodate the model size and data is the primary requirement.
H
Class label abbreviations
Table 5 provides abbreviations of class names in all evaluated datasets BloodMNIST, CIFAR-10, DermaMNIST and PathMNIST. Table 5: Class label abbreviations Full name Abbreviation Basophil Baso Eosinophil Eos Erythroblast EryBl Lymphocyte LymB Monocyte Mono Automobile Deer Horse Ship Truck
Auto Deer Horse Ship Truck
Basal cell carcinoma Dermatofibroma Melanoma Melanocytic nevi Vascular lesions
BCC DF Mel NV Vasc
Adipose Background Debris Lymphocytes
Adip BG Deb LymP
16
I
Performance on fine-tuning and fine-pruning defences
Figure 4 shows results for the fine-tuning and fine-pruning defences for source-target class pairs corresponding to those reported in Sec. 5.
Fine-tuning Fine-tuning (triggered)
Fine-pruning Fine-pruning (triggered) 1.0
0.8
0.8
0.8
Accuracy
0.6
Accuracy
1.0
Accuracy
1.0
0.6
0.4
0.6
0.4
0.2
0.4
0.2
0.2
(c) Ship-Truck
0.8
0.8
0.8
0.6
Accuracy
1.0
Accuracy
(b) Deer-Horse 1.0
Accuracy
(a) EryBl-Baso 1.0
0.6
0.4
0.6
0.4
0.2
0.4
0.2
(d) NV-Vasc
0.2
(e) Mel-DF
(f) BG-Deb
0.8
0.8
0.8
0.6
Accuracy
1.0
Accuracy
1.0
Accuracy
1.0
0.6
0.4
0.6
0.4
0.2
0.4
0.2
(g) LymP-Deb
0.2
(h) Baso-Eos
(i) LymB-Mono
0.8
0.8
0.8
0.6
Accuracy
1.0
Accuracy
1.0
Accuracy
1.0
0.6
0.4
0.6
0.4
0.2
0.4
0.2
(j) Auto-Truck
0.2
(k) Truck-Auto
(l) BCC-Mel
0.8
0.8
0.8
0.6
Accuracy
1.0
Accuracy
1.0
Accuracy
1.0
0.6
0.4
0.6
0.4
0.2 1
2
3
Epoch
4
(m) Mel-DF
5
0.4
0.2 1
2
3
Epoch
4
(n) Adip-BG
5
0.2 1
2
3
Epoch
4
5
(o) LymP-Deb
Figure 4: Accuracy over five epochs of (blue) fine-tuning and (green) fine-pruning defences for backdoored (a-g) ResNet18 and (h-o) ViT across four datasets. Results are reported for both clean inputs from all classes and triggered inputs from the source class. Shaded uncertainty bands represent the bounds of the 95% confidence interval. The full class names corresponding to the abbreviations are listed in App. H.
17
J
Performance comparison of defences on clean vs. triggered inputs
Table 6 compares the performance of defence methods pruning, parameter clipping and parameter noise injection on both clean and triggered inputs. Notably, the triggered results match those in Tab. 2, while this table additionally reports performance on clean inputs for comparison. Table 6: Performance comparison of defence methods pruning, parameter clipping and parameter noise injection on clean and triggered inputs for ResNet18 and ViT across four datasets. Clean denotes classification accuracy on unmodified inputs from all classes, while triggered indicates the attack success rate on triggered source-target class pairs. Uncertainty is reported as lower and upper bounds of the 95% confidence interval, with all values rounded to two decimal places. All hyperparameter configurations are provided in App. F, with full class names corresponding to the abbreviations listed in App. H. Pruning
Parameter clipping
Parameter noise
Model
Dataset
Source → Target
Clean
Triggered
Clean
Triggered
Clean
Triggered
ResNet18
BloodMNIST BloodMNIST CIFAR-10 CIFAR-10 DermaMNIST DermaMNIST PathMNIST PathMNIST
Baso → Eos EryBl → Baso Deer → Horse Ship → Truck NV → Vasc Mel → DF BG → Deb LymP → Deb
0.93 ± 0.01 0.93 ± 0.00 0.86 ± 0.00 0.87 ± 0.01 0.66 ± 0.01 0.63 ± 0.00 0.77 ± 0.02 0.80 ± 0.00
1.00 ± 0.00 0.58 ± 0.10 0.91 ± 0.01 0.93 ± 0.02 0.95 ± 0.02 0.94 ± 0.01 0.92 ± 0.09 0.99 ± 0.01
0.93 ± 0.00 0.96 ± 0.00 0.88 ± 0.00 0.90 ± 0.01 0.66 ± 0.01 0.70 ± 0.00 0.79 ± 0.02 0.84 ± 0.00
0.74 ± 0.04 0.54 ± 0.06 0.90 ± 0.01 0.92 ± 0.02 0.94 ± 0.02 0.97 ± 0.00 0.92 ± 0.08 0.35 ± 0.04
0.95 ± 0.01 0.94 ± 0.00 0.87 ± 0.00 0.88 ± 0.00 0.66 ± 0.01 0.69 ± 0.01 0.80 ± 0.01 0.82 ± 0.01
0.90 ± 0.03 0.53 ± 0.08 0.88 ± 0.02 0.86 ± 0.02 0.94 ± 0.02 0.96 ± 0.01 0.91 ± 0.10 0.72 ± 0.08
ViT
BloodMNIST BloodMNIST CIFAR-10 CIFAR-10 DermaMNIST DermaMNIST PathMNIST PathMNIST
Baso → Eos LymB → Mono Auto → Truck Truck → Auto BCC → Mel Mel → DF Adip → BG LymP → Deb
0.96 ± 0.00 0.95 ± 0.00 0.90 ± 0.00 0.90 ± 0.00 0.75 ± 0.02 0.74 ± 0.02 0.86 ± 0.00 0.87 ± 0.00
0.71 ± 0.06 0.31 ± 0.10 0.94 ± 0.07 0.90 ± 0.07 0.99 ± 0.00 0.99 ± 0.00 1.00 0.94 ± 0.01
0.96 ± 0.00 0.98 ± 0.00 0.93 ± 0.00 0.93 ± 0.00 0.72 ± 0.03 0.67 ± 0.03 0.94 ± 0.01 0.90 ± 0.00
0.90 ± 0.01 0.73 ± 0.08 0.95 ± 0.05 0.83 ± 0.08 0.81 ± 0.14 0.84 ± 0.07 1.00 ± 0.00 0.87 ± 0.01
0.95 ± 0.00 0.96 ± 0.00 0.91 ± 0.00 0.91 ± 0.00 0.70 ± 0.02 0.68 ± 0.01 0.89 ± 0.01 0.88 ± 0.01
0.90 ± 0.04 0.58 ± 0.12 0.92 ± 0.04 0.84 ± 0.06 0.77 ± 0.06 0.88 ± 0.03 1.00 0.71 ± 0.06
18
K
Backdoor layer weight distributions
Figure 5 shows the weight distribution in the backdoor layer for selected models and source-target class pairs. There is substantial overlap between the distributions of the clean and backdoored models, with the expected distinction that the backdoored distribution with spiked covariance exhibits heavier tails, with a slightly greater fraction of its mass concentrated away from the mode. The corresponding distributions of the remaining evaluated datasets from Sec. 5 exhibit a similar trend.
Clean model
0.2
0.1
0.0
0.1
Backdoor model
0.2
0.2
0.1
(a)
0.2
0.1
0.0
Weight value
0.0
0.1
0.2
0.1
0.2
(b)
0.1
0.2
0.2
(c)
0.1
0.0
Weight value (d)
Figure 5: Backdoor layer weight distributions of backdoored models compared to their clean counterparts for (a) ResNet18 on BloodMNIST with source class basophil and target class eosinophil, (b) ViT on the same dataset and class pair, (c) ResNet18 on DermaMNIST with source class melanocytic nevi and target class vascular lesions, and (d) ViT on DermaMNIST with source class melanoma and target class dermatofibroma.
L
Visual stealthiness of input trigger and the Maximum Mean Discrepancy metric
Figure 6 shows four examples of clean inputs alongside their triggered counterparts activating the backdoor. Figure 7 illustrates the influence of the scaling factor λ on the perceptibility of the trigger, showing that larger values of λ yield more visible perturbations. Stealthiness in terms of visual imperceptibility is quantified by the Maximum Mean Discrepancy (MMD) score [11], as the discrepancy between the original images and their triggered counterparts. MMD is a statistical measure determining whether two sets of samples are drawn from the same underlying distribution by computing the distance between their mean embeddings. Both the original and corresponding triggered images are passed through a pretrained InceptionV3 network to extract high-level feature representations denoted as X = {xi }ni=1 ⊂ Rp (original) and Y = {yj }nj=1 ⊂ Rp (triggered). A polynomial kernel function k : Rp × Rp → R defined as k(x, y) = (γxT y + c)d , with scaling factor γ = p1 , constant offset c = 1 and polynomial degree d = 3, computes the pairwise similarities of the feature embeddings. The biased empirical estimate of MMD is given by 19
MMD(X , Y) =
1 X 1 X 2 X k(x , x ) + k(y , y ) − k(xi , yj ), i j i j n2 i,j m2 i,j nm i,j
(7)
where the terms measure intra-set similarity of original features, intra-set similarity of triggered features, and cross-set similarity between original and triggered features, respectively. If the two sets originate from the same distribution, the MMD value approaches zero. Thus, a lower MMD score indicates higher similarity between the original and triggered image distributions, while larger values suggest stronger distributional shifts introduced by the trigger.
(a)
(b)
(c)
(d)
Figure 6: Original source class (left), triggered input (middle), and secret key (right, shown on a neutral background) images for different model-dataset configurations: (a) ResNet18 on BloodMNIST with source class basophil and target class eosinophil, (b) ViT on the same dataset and class pair, (c) ResNet18 on DermaMNIST with source class melanocytic nevi and target class vascular lesions, and (d) ViT on DermaMNIST with source class melanoma and target class dermatofibroma. Images are randomly selected and illustrate the visual effect of applying the secret key with scaling factor λ as used in experiments (Sec. 5).
0.35
MMD score
0.30
MMD Baseline = 0.00780 = 0.5 = 0.5
0.25 0.20 0.15 0.10 0.05 0.00
0
1
2
3
4
5
(a)
(b)
Figure 7: (a) Visibility of the trigger, quantified by the MMD score, as a function of the scaling factor λ. Results are shown for ResNet18 on BloodMNIST with source class basophil and target class eosinophil. The value of λ used in the final configuration giving results as presented in Sec. 5 is highlighted. A baseline MMD score is computed by splitting the original dataset into two halves. (b) Example input image from the source class with the same secret key weighted by (top left) λ = 0.01, (top right) λ = 0.5, (bottom left) λ = 2 and (bottom right) λ = 5.
20