A Self-Explainable Deep Architecture for Security Applications

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

A Self-Explainable Deep Architecture for Security Applications

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

cryptography, security, privacy, cybersecurity

Extended version of A Self-Explainable Deep Architecture for Security Applications (Accepted at the Conference on Game Theory and AI for Security (GameSec), 2026)

A Self-Explainable Deep Architecture for Security Applications Ananth Shreekumar Purdue University

Jyun-Jhu Syu Purdue University

Dongyan Xu Purdue University

arXiv:2608.05552v1 [cs.CR] 6 Aug 2026

Abstract Deep learning models have become integral to security applications due to their ability to model complex relationships in data and detect sophisticated threats. However, their complexity makes it difficult to understand how predictions are generated, posing significant challenges for interpretability, particularly in security applications where transparency is critical. Existing explanation methods, such as visual explanation techniques and post-hoc approaches, suffer from several limitations: reduced faithfulness due to local approximation errors, instability caused by reliance on randomness, and computational inefficiency that hinders real-time usage. To address these issues, we introduce XS EC, a self-explainable deep architecture developed for security applications. During training, XS EC uses a novel mask-based approach to extract informative sub-features from the data and learns prototypes, representative patterns that characterize each class. XS EC then leverages the prototypes in a dedicated similarity layer at test time to compute similarity scores and generates interpretable explanations without the need for post-hoc analysis. We evaluate XS EC across five diverse security scenarios, demonstrating its ability to achieve an average classification accuracy of 97.33% with minimal performance compromise. XS EC produces deterministic explanations for a fixed trained model and input and substantially reduces explanation latency compared with approximation-based and perturbation-based post-hoc methods. Through this effort, we extend the applicability of self-explainable AI to security applications, bridging the gap between deep learning performance and the need for explainability in critical scenarios.

Introduction

Deep learning models have achieved strong performance across security tasks such as malware detection, network intrusion detection, and binary code analysis. However, their lack of interpretability limits deployment in critical security settings [14,30], where analysts need faithful, understandable, and actionable explanations to support incident response.

Muslum Ozgur Ozmen Arizona State University

Z. Berkay Celik Purdue University To improve interpretability in security applications, prior work has largely adopted post-hoc explainable AI (XAI) methods, where explanations are generated after prediction by analyzing model outputs or internal behavior. These include approximation-based methods (e.g., LIME [39], SHAP [27], LEMNA [16]), gradient-based methods (e.g., Integrated Gradients [55], Grad-CAM [46], Saliency Maps [49]), and perturbation-based methods (e.g., Occlusion [66]). However, these methods often suffer from low fidelity due to approximation errors, high latency from repeated model queries or gradient computation, white-box access requirements, and instability (where the same input receives different explanations across runs) due to randomness. Recent security-specific methods improve surrogate fidelity [16], stabilize explanations [60], or adapt to evolving threats [63], but they remain post-hoc and inherit many of these limitations. Ante-hoc methods instead embed interpretability directly into the model architecture. Concept-based methods [23, 43, 64, 65] explain predictions using human-understandable concepts, but require carefully defined and often manually annotated concepts. Prototype-based methods [6, 11, 18, 22, 35] explain predictions through representative patterns, but existing methods largely target image data and rely on spatial locality and translational invariance, assumptions that do not hold for many security datasets. Other interpretable architectures, such as transformers [57] and neural additive models [1], provide useful inductive biases, but attention weights are not always faithful explanations of model behavior [4]. These limitations motivate a self-explainable architecture for security applications that jointly provides high-fidelity, sparse, stable, and low-latency explanations without relying on post-hoc analysis. We introduce XS EC, a self-explainable AI framework for security data that performs prediction and explanation jointly at inference time. To address the above requirements, XS EC learns masks to extract task-relevant sub-features, projects them into an embedding space, and compares them with class-specific prototypes to generate similarity scores. We use these scores both to compute class probabilities and, together with the learned masks, to generate

feature importance scores as an explanation. We evaluate XS EC on five security applications: PDF malware identification, phishing website detection, network intrusion detection, portable executable (PE) malware classification, and network attack classification. Across these tasks, XS EC maintains competitive classification performance while producing sparse, stable, and low-latency explanations. Compared with state-of-the-art post-hoc and ante-hoc XAI methods, XS EC achieves 100% explanation stability and reduces test-time explanation overhead by 3–10× on average. Our contributions are: • We introduce XS EC, a self-XAI model for security applications that uses prototype learning to jointly optimize predictive performance and explanation quality. • We propose a mask-based sub-feature extraction mechanism that enables XS EC to learn class-specific prototypes and generate human-understandable featureimportance explanations without post-hoc analysis. • We evaluate XS EC across five security applications and compare it against state-of-the-art XAI methods using established metrics for fidelity, sparsity, stability, and latency. Our results show that XS EC provides accurate predictions while producing stable, efficient, and highquality explanations.

Related Work

Explanation by Analysis. These methods generate explanations through a separate post-hoc analysis step. Approximation-based methods [16, 27, 39] approximate the target model with local explanation models [29], but this can reduce faithfulness to the original decision process, add overhead that limits real-time use, and introduce instability from random initialization [60]. In security settings, such limitations can misidentify attack-relevant features, delay response, or produce inconsistent explanations for identical inputs. Gradient-based methods [46, 49, 51, 53, 55] attribute importance using gradients of the output with respect to features. While they can provide detailed attribution signals and are often deterministic, their explanations may be difficult for operators to interpret when features are highly dependent or lack intuitive semantics. They also require white-box access to model parameters, which can be impractical for proprietary or obfuscated security models, and often require domain-specific adaptation to produce human-interpretable feature attributions. Perturbation-based methods [12,66,68] explain predictions by modifying input features or intermediate activations and observing changes in the output. Although conceptually simple, they require repeated model evaluations and can thus incur high computational cost. Depending on the perturbation

strategy, they may produce inconsistent explanations due to randomized perturbations. Explanation by Design. In contrast to post-hoc methods, ante-hoc methods build interpretability into the model architecture. SENN [2] formalizes self-explaining models around explicitness, faithfulness, and stability, while Neural Additive Models [1] provide inspectable feature-wise effect functions. Transformers [57] are often interpreted through attention weights, but attention may not faithfully explain model behavior [4]. TabNet [3] uses sequential attention masks for tabular prediction, and InterpreTabNet [48] further improves the interpretability of such masks by encouraging sparsity and diversity; however, these methods are attention-based rather than prototype-based. Concept-based methods [23, 43, 64, 65] explain predictions using abstract, human understandable concepts learned from data annotated with labels and concepts. TabCBM [10] extends concept-based self-explaining models to tabular data by learning high-level concept explanations with partial or no concept supervision. However, concept-based methods depend on defining a sufficiently complete and interpretable concept set, and supervised variants require time-consuming, task-specific concept annotations [9]. Prototype-based methods explain predictions using representative patterns learned from data. Early prototypelayer models use distances to learned latent prototypes as part of the classifier [25], and ProtoPNet-style architectures [6, 35, 40, 41, 58] learn class-specific prototype parts for image recognition using convolutional feature extractors [24]. LEXNet [11] applies a related explainable-by-design prototype architecture to Internet traffic classification, but still relies on a CNN-based design. More recently, ProtoGate [20] combines prototype-based prediction with global-to-local feature selection, and MEDIC [19] learns prototype parts using trainable feature patching and binary or discretized feature subsets. Unlike ProtoGate and MEDIC, which are developed primarily for biomedical tabular data, XS EC targets featurevector security tasks and combines prototype-specific masks, class-specific prototype matching, and deterministic featureimportance explanations in a single architecture.

Problem Statement and Requirements

We consider a tabular security dataset D = {x(i) , y(i) }Ni=1 , where x(i) ∈ Rd is the ith sample and y(i) ∈ {1, . . . ,C} is its class label. Each dimension of x(i) corresponds to a measurable security feature. Our goal is to generate explanations by identifying the features that contribute most to a model prediction, such as whether a sample is malicious or benign. Achieving this goal for tabular security data imposes several design requirements, which we summarize below. (C1 ) Sub-feature Extraction. A key challenge is learning prototypes from informative sub-features of tabular security

data. Existing prototype-based methods (e.g., ProtoPNet [6]) rely on CNNs to extract patterns. However, security features generally lack spatial locality and translational invariance, making CNN-based pattern extraction unsuitable. We therefore require a mechanism for extracting meaningful feature subsets without assuming image-like structure. (C2 ) Transparency of the Explanation Method. Security explanations must be transparent enough for analysts to inspect, trust, and act upon. Complex or non-intuitive explanations can reduce practical utility; for example, gradient-based methods may produce abstract attributions that are difficult for operators to interpret. Thus, the explanation process should be simple, direct, and aligned with the model’s prediction mechanism. (C3 ) Ability to Generate Importance Scores. Many XAI methods, including LIME, SHAP, and LEMNA, explain predictions using feature-importance scores, while image-based methods often highlight salient regions or prototype-matched patches. For tabular security data, however, visualizing prototypes directly is not meaningful because prototypes are real-valued vectors in an embedding space. The model must therefore convert its internal prototype-based reasoning into human-understandable feature-importance weights. (C4 ) Multi-Objective Explanation Synthesis. Effective security explanations must jointly satisfy four objectives: (1) high fidelity: explanations reflect the model’s true prediction process; (2) sparsity: only a small set of features is marked important; (3) stability: identical inputs receive consistent explanations across runs; and (4) low latency: quick explanations can support real-time use. These requirements are especially important in security, where unstable or slow explanations can delay response or mislead analysts. Unlike image tasks, where explanations may cover broad regions and tolerate small variations, security applications often require concise, consistent, feature-level evidence. We design XS EC to address these objectives simultaneously.

Methodology

We introduce XS EC, a novel prototype-based self-explainable framework designed for security applications. It is composed of four components as shown in Figure 1: (1) a mask generator NN M , (2) a feature encoder NN f , (3) a prototype layer gP , and (4) a fully connected layer h that outputs classification scores. XS EC also learns a set of prototypes p. Given C classes and k prototypes per class, XS EC learns C × k prototypes and generates C × k masks. The networks M , f , and h are not constrained in their architecture. Each can be instantiated with any neural architecture whose input and output dimensions match the required dimensions. XS EC uses masks to extract informative sub-features from tabular security data (C1) and a prototype layer to produce feature-importance explanations through a transparent pre-

Figure 1: An overview of XS EC. The input is the sample x, and the outputs are the importance weights w and the class probabilities c. diction mechanism (C2, C3). A multi-objective loss guides training to satisfy the required explanation properties (C4). Given an input sample, XS EC first generates binary masks m using the mask generator M ( 1 ), where each mask has the same dimension as the input. These masks select sub-features, which are then encoded by f into an embedding space ( 2 ). The prototype layer gP computes similarity scores between these embeddings and their corresponding prototypes ( 3 ). Finally, the fully connected layer h maps the similarity scores to class probabilities using softmax activation [5] ( 4 ). In the explanation procedure ( A , B ), XS EC leverages the masks and the computed similarity scores s to generate importance weights w for each feature such that a higher weight value represents a more important feature.

4.1

Sub-feature Extraction

Given a sample x(i) ∈ Rd , we extract sub-features and compute their similarity to learned prototypes. A sub-feature is a subset of jointly informative features that indicates a class. For example, in phishing detection, the co-occurrence of links and images may indicate phishing even when either feature alone does not. XS EC operates on security datasets where each data point is a vector of real numbers. Unlike images, such data generally lack spatial locality and translational invariance, making CNNbased feature extraction unsuitable. To extract sub-features, XS EC uses a mask generator M . Given an input vector of length d, M generates a set of masks {m j }C×k j=1 . Each mask m j extracts a sub-feature vector from x(i) through element-wise multiplication: (i) q j = x(i) ⊙ m j (1) Binary Masks for Analyst-Interpretable Explanations. A sigmoid activation restricts the output of M to [0, 1], but this is insufficient since we require each element to be in {0, 1} for intuitive explanations. For example, multiplying the number

of links in a PDF by 0.5 would select half of its value rather than indicate whether the feature is relevant. We therefore encourage each feature to be selected entirely or excluded by introducing the binary-mask loss

4.3

Last Layer Classification

The final layer h maps similarities s(i) to class probabilities: c(i) = Softmax(h(s(i) ))

(7)

C×k

Lbin = ∑

∥mi ⊙ (1 − mi )∥22

(2)

i=1

Each element-wise product is minimized at 0 and 1, thereby encouraging binary masks while preserving differentiability during training. Learning Sparse Sub-features. Concise explanations identify only a small subset of the input features and are humaninterpretable. Accordingly, each prototype represents an embedding of a compact sub-feature rather than the entire input. We encourage such sparsity in explanations by using the sparsity loss 1 C×k Lspar = (3) ∑ ∥mi ∥22 C × k i=1 For binary masks, ∥mi ∥22 equals the number of selected features; minimizing Lspar therefore encourages each mask to select fewer features. To discourage different prototypes from relying on redundant feature subsets, we introduce the mask-similarity loss

Because h receives only prototype-similarity scores, predictions are determined entirely by prototype matches, directly linking classification and explanation. The predicted class is (i) o(i) = arg max j c j . Figure 2 illustrates XS EC’s classification process with three prototypes per class (k = 3). The prototypes and masks are learned during training and fixed thereafter. Thus, all test samples use the same prototype-specific masks and prototypes; neither is generated per sample.

4.4

Training Procedure

We divide XS EC’s training process into two phases: Phase 1: Joint Learning. In this phase, the mask generator, feature encoder, and prototypes are jointly trained to learn informative masks, prototypes, and embeddings while maintaining predictive performance. We use cross-entropy loss

C×k C×k

mi · m j Lsim = ∑ ∑ i=1 j=i+1 ∥mi ∥∥m j ∥

(4)

which is the sum of pairwise cosine similarities among the masks. Minimizing it results in masks that are dissimilar.

4.2

Similarity Score Computation (i)

The feature encoder f maps each sub-feature vector q j to an embedding. The prototype layer then computes its similarity to the corresponding prototype p j : (i)

(i)

s j = gP j ( f (q j )) = log

∥ f (q(i) ) − p j ∥2 + 1 j 2 (i) ∥ f (q j ) − p j ∥22 + ε

Lxe = −

(5)

where 0 < ε < 1 prevents division by zero. This score decreases with the distance between the sub-feature embedding and its prototype; therefore, a higher score indicates a closer match. To learn a useful embedding space, we minimize the cluster loss 1 N 2 Lcls = ∑ min ∥ f (q(i) (6) j ) − p j ∥2 N i=1 j:p j ∈Py(i) where Py(i) denotes the prototypes associated with the true class of x(i) . This loss encourages at least one class-associated sub-feature embedding to be close to its corresponding prototype. We enforce Lipschitz continuity of f following [15], limiting changes in the learned embeddings (and consequently the explanations) under small input perturbations.

1 N C (i) ∑ ∑ log(c j ) · 1(y(i) = j) N i=1 j=1

(8)

(i)

where c j is the predicted probability of class j for x(i) . During this phase, the last-layer weights are fixed to guide prototype learning. Let Wh denote the weight matrix of h, jl where Wh connects the similarity score of prototype p j to the logit for class l. The weight is positive when p j belongs to class l and negative otherwise. Thus, matching a prototype increases its associated class logit while decreasing the logits of other classes. We set ( 1 if p j ∈ Pl jl Wh = (9) −0.5 if p j ̸∈ Pl The negative weights ensure that when a sample matches a prototype from one class, its scores for competing classes decrease. This helps the model distinguish between classes more clearly. We train the mask generator, feature encoder, and prototypes jointly while keeping the last layer h fixed. The joint objective is

L joint = λ0 Lxe + λ1 Lspar + λ2 Lcls + λ3 Lbin + λ4 Lsim (10) where the coefficients λ are tunable hyperparameters. Phase 2: Last Layer Fine-tuning. After joint training, we freeze the mask generator, feature encoder, and prototypes and optimize only the last layer h. To improve classification while

Mask generator

Sample Masks x m

Sub-features Feature q encoder

Similarity scores s

Prototypes p

𝑓 3

Sub-feature embeddings 𝑓(𝐪)

Last layer ℎ

Similarity layer 𝑔𝐏

Class Prob.

Figure 2: Illustration of XS EC classifying a five-dimensional sample using a seven-dimensional embedding and three prototypes per class (k = 3). Colors identify input features; embeddings and prototypes are uncolored because they reside in the learned embedding space. Lighter shading in the similarity scores indicates greater prototype-embedding similarity.

Masks m

Algorithm 1 Explanation procedure for XS EC.

Similarity scores s Importance weights w

Require: Sample x, n ∈ Z : n ∈ [1, k] Ensure: Predicted class o, importance scores w 1: q ← x ⊙ m ∥ f (q)−p∥22 +1 2: s = gP ( f (q)) = log 2 ∥ f (q)−p∥ +ε 2

A 0

3: c = Softmax(h(s)) 4: o = arg max j c j 5: Let I be the indices of top n similarity scores corresponding to

class o.

Figure 3: The explanation process of XS EC. Here, the number of prototypes per class k = 3 and the number of highest similarity scores to consider n = 2.

6: w = embeddings mi ∑Sub-feature i∈I si ·𝑓(𝐪) 7: return o, w p1 𝐏1

suppressing connections between prototypes and unrelated classes, we use C

Lreg = ∑ ∑ |Whjl |

(11)

l=1 p j ∈P / l

The fine-tuning objective is

Llast = Lxe + λLreg

(12)

𝑔 𝑔 Algorithm 1 details this procedure.𝑔 Given n ∈ {1, . . . , k}, Similarity XS EC extracts the sub-features (line 1), computes their scores s prototype-similarity scores (line 2), and obtains the predicted class (lines 3–4). It then selects the n highest-scoring prototypes associated with that class (line 5) and computes w as a similarity-weighted sum of their masks (line 6, A , B ). 𝐏𝟐

𝐏6

Deterministic Explanations. For a fixed model and input, Algorithm 1 is deterministic and therefore always produces the same explanation. This avoids the run-to-run variability caused by stochastic post-hoc explanation procedures.

where λ controls the regularization strength. By shrinking off-class weights toward zero, Lreg encourages predictions to rely on similarities to prototypes of the predicted class rather than on the absence of similarities to other-class prototypes. Thus, the model primarily relies on positive evidence in its predictions.

Real-Time Explanations. Prediction and explanation share a single forward pass. Explanations require selecting the highest-scoring prototypes and aggregating their masks, without training surrogate models or perturbing inputs.

4.5

We evaluate XS EC on five security datasets and show that it maintains competitive classification performance, and that its explanations are high-fidelity, sparse, stable, and low-latency.

Explanation Procedure

XS EC generates a feature-importance vector w, where larger values indicate greater importance. Figure 3 shows the explanation process in detail for k = 3 and n = 2. Lighter shades represent higher similarity scores. The figure assumes that the sample has been classified as class 1, and therefore it considers the highest 2 similarity scores for class 1 and their corresponding masks to generate importance weights. This is consistent with our requirement that higher similarity scores translate to higher feature importance weights.

5.1

Experiments

Experimental Setup

Datasets. We evaluate XS EC on five security datasets: PDF malware identification [52, 54], phishing website detection [7], network intrusion detection [45], PE malware classification [62], and network attack classification [56].

These datasets cover binary and multi-class tasks with 2–5 classes, 10K–20K samples, and 38–135 features, and have been used in prior security-XAI studies [16, 47, 60]. The PDF malware dataset contains 135 structural PDF features from benign and malicious files; the phishing dataset contains 38 URL/webpage features from phishing and legitimate websites; the network intrusion dataset contains 39 NetFlow features sampled from benign and attack traffic; the PE malware dataset uses BODMAS malware-family features, retaining families with at least 3,000 samples and the top 100 ANOVAranked non-constant features; and the network attack dataset uses NSL-KDD flow features for benign traffic and attack classes with at least 3,000 samples. For all datasets, we use an 80/20 stratified train-test split and standardize features to zero mean and unit variance. We open-source preprocessing scripts and dataset subsets. Please see Appendix A for details of the datasets. Baselines. We compare XS EC with representative post-hoc and ante-hoc XAI methods. The post-hoc baselines include (1) approximation-based methods, LIME [39] and SHAP [27]; the security-specific improvement LEMNA [16]; (2) gradientbased methods, Integrated Gradients (IG) [55], Guided GradCAM (GGC) [46], and Saliency Maps (SM) [49]; and (3) a perturbation-based method, Occlusion (Occl.) [66]. The ante-hoc baselines include ProtoPNet (PPN) [6] and a Transformer architecture (Tran.) [57]. Since ProtoPNet is designed for images, we adapt it using 1D convolutions and derive feature importance using gradients. For the Transformer, we use averaged attention weights as feature-importances. We additionally compare with xNIDS [60] on network intrusion detection. Evaluation Metrics. We measure classification performance using accuracy, precision, recall, and false positive rate (FPR), and explanation quality using fidelity, sparsity, stability, and latency metrics. Fidelity. Fidelity measures whether an explanation identifies features that actually drive the model’s prediction. Following prior work [16,42,60], we use three feature-perturbation tests. For a sample x, let Fx denote the top features selected by an explanation method, with |Fx | being the explanation size. 1. Synthetic Test. We preserve the selected features Fx in x and randomize all remaining features. A faithful explanation should preserve the original prediction under this perturbation. This test is analogous to LeRF [42]. 2. Feature Augmentation Test. We insert the selected features Fx from x into a sample from a different class to obtain xs . A faithful explanation should cause xs to be classified as x’s original class. 3. Feature Deduction Test. We randomize the selected features Fx in x while preserving all other features. A faithful explanation should change the original prediction. This test is analogous to MoRF [42].

Table 1: Classification performance across datasets. Classifier

XS EC

MLP / LSTM

CNN

ProtoPNet

Transformer

Metric† Acc. Pre. Rec. FPR Acc. Pre. Rec. FPR Acc. Pre. Rec. FPR Acc. Pre. Rec. FPR Acc. Pre. Rec. FPR

Dataset PDF Website Network PE Network Malware Phishing Intrusion Malware Attack 98.95% 93.85% 99.15% 94.79% 99.92% 99.30% 93.03% 98.91% 95.30% 99.92% 98.60% 94.80% 99.40% 95.38% 99.92% 0.70% 7.10% 1.10% 1.32% 0.03% 99.85% 96.10% 99.42% 95.72% 99.92% 99.90% 96.19% 99.06% 96.13% 99.92% 99.80% 96.00% 99.80% 96.21% 99.92% 0.10% 3.80% 0.95% 1.09% 0.03% 99.85% 96.45% 99.45% 96.53% 99.96% 100.0% 96.87% 99.06% 96.99% 99.96% 99.70% 96.00% 99.85% 96.93% 99.96% 0.00% 3.10% 0.95% 0.89% 0.01% 99.80% 96.30% 99.47% 96.56% 100.0% 100.0% 96.30% 99.16% 96.89% 100.0% 99.60% 96.30% 99.80% 96.94% 100.0% 0.00% 3.70% 0.85% 0.88% 0.00% 97.55% 94.40% 96.85% 95.70% 98.00% 98.57% 92.20% 94.58% 95.94% 98.00% 96.50% 97.00% 99.40% 96.13% 98.0% 1.40% 8.20% 5.70% 1.10% 0.67%

† Acc.: accuracy, Pre.: precision, Rec.: recall, and FPR: false positive rate.

For precision, recall, and FPR, mean value is reported for multi-class datasets.

Sparsity. Sparse explanations are easier to inspect because they identify only a small set of important features [59]. We measure sparsity using the Mean Around Zero (MAZ) curve: after normalizing absolute feature-importance scores to [0, 1], MAZ measures how much score mass lies near zero. A steeper curve near zero indicates a sparser explanation. Stability. Stability measures whether an explanation method identifies the same important features across runs [59]. For two runs with top-N feature sets Ti and T j , we compute |Ti ∩ T j |/N; values closer to 1.0 indicate more stable explanations. Implementation. We implement XS EC in PyTorch [36] with the Adam optimizer [21]. We use k = 10 prototypes per class for all datasets. The feature encoder f is an MLP for the PDF malware, phishing website, network intrusion, and PE malware datasets, and an LSTM for the network attack dataset to model temporal relationships. We select loss coefficients, learning rates, batch sizes, embedding dimensions, and network widths using K-fold cross-validation. Experiments are run on a laptop with an AMD Ryzen 7 CPU, an NVIDIA GeForce RTX 3050Ti GPU, and 16GB RAM. For the baselines, we use published implementations when available [17, 26, 37, 38, 61], and include preprocessing scripts and baseline configurations in our released code1 . XS EC hyperparameters are reported in Appendix B.

5.2

Classification Performance

We first evaluate whether XS EC preserves classification performance despite its architectural constraints. For each dataset, 1 https://github.com/ashreeku/XSec

Synthetic PCR (%)

100

XSec (n = 3) PDF Malware Identification

100

10 30 50 70 90 110 130 Feature Deduction Feature Augmentation PCR (%) PCR (%)

LIME SHAP IG Website Phishing Detection

GGC Occl. SM Network Intrusion Detection

100

PPN Tran. PE Malware Classification

50 5

LEMNA xNIDS Network Attack Classification 100

100

10 30 50 70 90 110 130

100 75 50 10 30 50 70 90 110 130 |Fx|

5 80 60 40

60 15

|Fx|

75 50 25

0 10

|Fx|

50 70 |Fx|

75 50 25

|Fx|

Figure 4: Comparison of XS EC with baseline explanation methods under three fidelity tests across all datasets. Higher PCR is better for the synthetic and feature augmentation tests; lower PCR is better for the feature deduction test. we compare XS EC against tuned neural baselines: an MLP for PDF malware, phishing website, network intrusion, and PE malware classification; an LSTM for network attack classification; and CNN, Transformer, and ProtoPNet models across all datasets. Table 1 shows that XS EC achieves competitive performance across all tasks. Compared with the strongest neural baselines, XS EC incurs only a small accuracy loss on PDF malware, phishing website, and PE malware classification, while achieving near-identical performance on network intrusion and network attack classification. These results show that XS EC provides self-explainable predictions with minimal compromise in predictive performance.

5.3

Explainability Results

Fidelity. For each test sample, we apply the three fidelity tests and report the positive classification rate (PCR) as a function of |Fx |. PCR is the fraction of modified samples classified as the original prediction. Higher PCR indicates better fidelity for the synthetic and feature augmentation tests, while lower PCR indicates better fidelity for the feature deduction test. Figure 4 shows that XS EC provides consistently strong fidelity across datasets. In the synthetic test, XS EC performs best on PDF malware, network intrusion, and network attack classification, and remains comparable on phishing and PE malware. In the feature augmentation test, XS EC is strongest on network intrusion and competitive elsewhere. In the feature deduction test, XS EC remains comparable to other methods, although Occlusion performs best. XS EC consistently outperforms LIME, SHAP, Saliency Maps, ProtoPNet, and Transformer. Sparsity. Figure 5 shows the MAZ curves for each ex-

Table 2: Area Under the Curve of the MAZ sparsity curves. A score closer to 1.0 along with a steep rise near 0 in Figure 5 indicates sparser explanations. PDF Malware Phishing Website Network Intrusion PE Malware Network Attack

XS EC LIME SHAP LEMNA

0.909 0.785

0.942

0.970

0.904 0.952 0.841 0.859 0.858 0.884

0.793 0.704

0.901

0.935

0.877 0.909 0.770 0.743 0.795 0.904

0.819 0.741

0.886

0.893

0.883 0.890 0.796 0.805 0.799 0.863

0.912

−†

0.892 0.956 0.813 0.838 0.795 0.921

0.874

−†

0.891 0.928 0.911 0.726 0.760 0.793

0.849 0.761 0.798 0.809

GGC Occl.

PPN Tran.

† LEMNA does not support multi-class datasets.

planation method, and Table 2 reports their AUC values. Curves that rise sharply near zero indicate sparser explanations. XS EC consistently produces steep MAZ curves and high AUC scores across datasets, indicating that its explanations place importance on a small set of features. Some baselines achieve higher sparsity than XS EC on individual datasets, but sparsity must be interpreted alongside fidelity: an explanation that is sparse yet does not reflect the model’s decision is not useful. For example, LEMNA, SHAP, GGC, Transformer, and xNIDS sometimes produce sparser explanations, but they generally show lower fidelity in Figure 4. Conversely, Occlusion is less sparse but performs strongly in the feature deduction test. Overall, XS EC provides a favorable balance between sparsity and fidelity across datasets. Stability. We evaluate stability by running each explanation method with two different seeds and measuring the average overlap between the top-10 features selected in each run. The results are reported in Table 3. XS EC is deterministic for a fixed trained model and input, and therefore achieves a

MAZ

1.0

XSec (n = 3) PDF Malware Identification

1.0

LIME SHAP IG Phishing Website Detection

1.0

GGC Occl. SM Network Intrusion Detection

1.0

PPN Tran. PE Malware Classification

LEMNA xNIDS Network Attack Classification 1.0

0.5

0.00.00 0.25 0.50 0.75 1.00 Interval Size

Figure 5: The Mean Around Zero (MAZ) sparsity curves. A steeper curve closer to 0 indicates a sparser explanation method.

Table 3: Stability of XAI methods. A score closer to 1.0 indicates higher stability. XS EC LIME SHAP LEMNA IG GGC Occl. SM PPN Tran. PDF Malware Phishing Website Network Intrusion PE Malware Network Attack

1.0

0.116

0.170

0.381

1.0

0.559

0.771

0.704

1.0

0.419

0.532

0.740

1.0

0.192

0.229

−

1.0

0.560

0.586

−

1.0

Table 5: The effect of number of prototypes per class (k) on XS EC performance. Dataset PDF Malware

Website Phishing

Network Intrusion

Table 4: Mean and standard deviation of latency in seconds over three runs. PDF Malware Phishing Website Network Intrusion PE Malware Network Attack

XS EC 3.37 ±0.18 1.38 ±0.14 4.59 ±0.27 6.60 ±0.77 5.22 ±0.35

LIME 193.85 ±11.11 18.19 ±0.30 40.35 ±4.64 357.03 ±18.02 28.11 ±0.24

SHAP 401.68 ±14.02 42.82 ±0.53 293.69 ±37.21 1871.55 ±34.23 171.16 ±2.59

LEMNA 2296.84 ±9.37 516.77 ±8.65 1060.75 ±44.99 − −

IG 10.86 ±0.29 10.60 ±0.10 27.63 ±0.57 21.15 ±0.33 13.45 ±0.20

GGC 2.60 ±0.09 2.17 ±0.02 5.46 ±0.07 4.76 ±0.03 2.66 ±0.10

Occl. 74.73 ±0.16 20.24 ±0.34 42.17 ±0.27 111.51 ±0.12 54.37 ±0.46

SM 0.74 ±0.01 0.70 ±0.01 1.49 ±0.05 1.54 ±0.06 2.40 ±0.01

PPN 18.75 ±1.91 3.29 ±0.01 6.39 ±0.01 150.27 ±3.00 1.97 ±0.07

Tran. 7.52 ±0.03 6.43 ±0.05 12.87 ±0.76 14.25 ±0.57 6.76 ±0.15

PE Malware

Network Attack

Metric Acc. Pre. Rec. FPR Acc. Pre. Rec. FPR Acc. Pre. Rec. FPR Acc. Pre. Rec. FPR Acc. Pre. Rec. FPR

Number of prototypes per class k=3 k=5 k=20 k=30 50.00% 90.05% 99.70% 99.80% 50.00% 85.47% 100.00% 100.0% 100.00% 96.50% 99.40% 99.60% 100.00% 16.40% 0.00% 0.00% 50.00% 56.75% 95.30% 95.15% -† 54.61% 94.50% 93.37% 0.00% 80.00% 96.20% 97.20% 0.00% 66.50% 5.60% 6.90% 62.65% 86.05% 99.33% 99.47% 60.08% 78.19% 99.01% 99.11% 75.40% 100.00% 99.65% 99.85% 50.10% 27.90% 1.00% 0.90% 23.25% 23.08% 95.13% 95.23% 4.651% 4.617% 95.63% 95.62% 20.00% 20.00% 95.69% 95.80% 20.00% 20.00% 1.24% 1.21% 25.00% 99.25% 99.83% 99.87% 6.250% 99.25% 99.83% 99.88% 25.00% 99.25% 99.83% 99.87% 25.00% 0.25% 0.056% 0.042%

† − means no samples were classified positive; thus, precision is unde-

fined.

5.4 stability score of 1.0. The deterministic baselines—IG, GGC, Saliency Maps, Occlusion, ProtoPNet, and Transformer—also achieve perfect stability. In contrast, post-hoc methods such as LIME, SHAP, LEMNA, and xNIDS produce lower stability because their explanations depend on stochastic perturbation or sampling procedures. Latency. We measure test-set explanation time by running each method on test samples individually, without batching or parallelization, under the same experimental setup. Each experiment is repeated three times, and Table 4 reports the mean and standard deviation. XS EC has low test-time latency because prediction and explanation share a single forward pass. In contrast, approximation-based methods such as LIME, SHAP, and LEMNA require repeated perturbation and model evaluation, resulting in substantially higher latency. xNIDS is the slowest method on network intrusion detection, requiring 7,200 ± 600 seconds.

Hyperparameter Sensitivity

We perform experiments for hyperparameter sensitivity of XS EC with respect to the number of prototypes per class (k) and the number of similarity scores (n). Effect of Number of Prototypes. To study the effect of the number of prototypes per class (k) on model performance and the fidelity of explanations, we train several models with identical parameters except for the value of k. We set n = 3 for this evaluation. The performance of these models is given in Table 5. As expected, the model is unable to capture the important prototypes for low values of k. Performance increases with more prototypes per class and plateaus at some value of k since extra prototypes do not always encode additional useful information for classification. Using a very low number of prototypes per class (k = 3), XS EC does not have enough capacity to learn effective prototypes, leading to a drastic decrease in performance. However,

Feature Deduction Feature Augmentation PCR (%) PCR (%)

Synthetic PCR (%)

100

PDF Malware Identification

Website Phishing Detection

k = 10 k = 20 k = 30 Network Intrusion Detection 100 100

10 30 50 70 90 110 130

PE Malware Classification

50 5

100 50

100

10 30 50 70 90 110 130

10 30 50 70 90 110 130 |Fx|

|Fx|

75 50 25

Network Attack Classification

60 40 10

50 70 |Fx|

|Fx|

Figure 6: The effect of hyper-parameter k (the number of prototypes per class) on the fidelity of explanations. at k = 20, the model is able to perform effectively on all three metrics. We also show the impact of k on explanation fidelity, given in Figure 6. For the PDF malware dataset, explanation fidelity does not necessarily improve with increasing k; in fact, it suffers slightly since more noise is built into the explanation process. The same observations apply to the phishing website and PE malware datasets, in which the fidelity is highest with k = 10 and decreases for higher values of k. This effect is most prominent for the network intrusion and network attack datasets, as XS EC performs quite poorly with higher values of k. This is because the feature dimensionality is low, and learning many prototypes with very few features makes the training process result in prototypes that are unable to represent a useful embedding space. Effect of Number of Similarity Scores. The hyper-parameter n controls the number of highest similarity scores to consider while generating an explanation. The value of n may change the explanations generated as the importance scores are a sum of masks weighted by similarity scores. We train a model with k = 10 and study the effect of n on explanation fidelity. As seen for the PDF malware dataset, all three fidelity tests suffer if the value of n is very low (e.g., 1). Intuitively, this is expected since a single prototype is unable to capture enough information to completely explain a sample. However, as n increases, it reaches a point where all information has already been acquired, and the use of additional similarity scores and masks only adds noise to the explanation. It is important to note that the fidelity of the explanations does not suffer for high values of n until very high values of |Fx | are used. This is consistent with intuition, as the first few features are assigned very high importance scores and are always dominant contributors. When the value of |Fx | is high, a lot of noise is injected into the tests and fidelity suffers. For the phishing website and the PE malware datasets, it

can again be seen that a single prototype is not effective in being able to generate good explanations. Moreover, the figure shows that explanation fidelity is best with n = 3, which shows that more prototypes capture useful information. We can again see here that the fidelity of the explanation decreases for large values of n when large explanation set sizes (high value of |Fx |) are used. This is consistent with the observation for the PDF malware dataset. Interestingly, even though the feature dimensionality is low for the network intrusion and network attack datasets, the fidelity of the explanations is low for n = 1 at low values of |Fx |. However, the performance rapidly increases as more features are considered. As in other datasets, n = 3 performs the best, and fidelity does not improve much even with more prototypes being considered.

5.5

Ablation Study

We ablate the explanation-specific loss terms in the joint objective (Equation 10) to evaluate their effect on explanation quality. Figure 8 shows the effect on fidelity. Removing the sparsity loss Lspar , binary-mask loss Lbin , or cluster loss Lcls degrades fidelity on several datasets, indicating that these terms are important for producing reliable prototype-based explanations. The feature-augmentation curves for the model without Lsim are omitted because this ablation collapses to predicting a single class, making the test inapplicable. Figure 9 shows the effect on sparsity. As expected, removing Lspar reduces explanation sparsity. We also observe a collapse case on the phishing website dataset when Lsim is removed: the model learns all-zero masks, yielding zero importance scores for all samples. This behavior is consistent with the sparsity and binary-mask losses favoring zero-valued masks when no diversity term discourages identical masks. Stability is 1.0 for all ablations because XS EC’s explanation procedure is deterministic for a fixed model and input.

Synthetic PCR (%)

100

n=3 Website Phishing Detection

PDF Malware Identification 90 80 70

Feature Deduction Feature Augmentation PCR (%) PCR (%)

10 30 50 70 90 110 130 100 50

100

50 5

80 60 5

|Fx|

50 10

5 100

50 5

Network Attack Classification

80 10

100

60 10 30 50 70 90 110 130 |Fx|

100 90

50 15

n = 10 PE Malware Classification

80 5

10 30 50 70 90 110 130 80

n=1 n=5 n=7 Network Intrusion Detection 100 100

50 70 |Fx|

|Fx|

Figure 7: The effect of hyper-parameter n (the number of similarity scores to consider) on the fidelity of explanations.

Feature Deduction Feature Augmentation PCR (%) PCR (%)

Synthetic PCR (%)

100

PDF Malware Identification

100

0 10 30 50 70 90 110 130

No ablation Website Phishing Detection

bin

100

sim

Network Intrusion Detection

50 5

cls

spar

PE Malware Classification

100 50

0 10

100

0 10 30 50 70 90 110 130

0 10

100

0 10 30 50 70 90 110 130 |Fx|

|Fx|

Network Attack Classification

0 10

50 70 |Fx|

|Fx|

Figure 8: Effect of ablating loss terms on explanation fidelity.

MAZ

1.0

PDF Malware Identification

1.0

No ablation Phishing Website Detection

bin

1.0

sim

Network Intrusion Detection

cls

1.0

spar

PE Malware Classification

1.0

Network Attack Classification

0.5

0.00.00 0.25 0.50 0.75 1.00 Interval Size

Figure 9: Impact of removing each loss term on explanation sparsity.

5.6

Scalability over Feature Dimensionality

We perform a scalability analysis of XS EC over the number of features. We use the PE malware dataset for this experiment as it has a high feature dimensionality to perform the analysis. We repeat our fidelity and sparsity experiments with 100, 200, and 500 features selected by their high ANOVA F-values.

The model architecture remains the same for each feature dimensionality, except for the number of prototypes per class k, which is set to 10, 20, and 50. The results are given in Table 6 and Figure 10. As expected, we notice a slight increase (≈ 1.5%) in performance when the number of features increases from 100 to 500. Yet, explanation fidelity of XS EC improves with fewer features. Further,

0 0.00 0.25 0.50 0.75 1.00 |Fx| d

(a)

50 0 0.00 0.25 0.50 0.75 1.00

200

500 Feature Deduction Test

100 50

0 0.00 0.25 0.50 0.75 1.00

|Fx| d

1.0 MAZ

100

100 Feature Augmentation Test

PCR (%)

100

Synthetic Test

|Fx| d

(b)

(c)

Sparsity

0.5 0.00.00 0.25 0.50 0.75 1.00 Interval Size (d)

Figure 10: Scalability analysis of the explanation fidelity (a, b, c) and sparsity (d) of XS EC. The x-axis for (a), (b), (c) is the fraction of features modified or removed.

Table 6: Classification performance of XS EC on the PE Malware dataset with varying feature dimensionality.

Accuracy Precision Recall FPR

Feature Dimensionality 100 200 500 94.79% 95.62% 96.36% 95.30% 96.09% 96.79% 95.38% 96.14% 96.77% 1.33% 1.12% 0.93%

we notice that sparsity is similar for all three models. However, this is not strictly true in terms of human interpretability. This is because an explanation of 10 features for the model with 100 features is considered to be as sparse as an explanation of 50 features for the model with 500 features. Yet, 10 features are far more interpretable by humans than 50 features. Based on the above observations, we recommend using dimensionality reduction techniques before training XS EC. By doing this, the sparsity of explanations improves with little effect on the fidelity or classification accuracy.

5.7

Case Study

We provide a qualitative case study to illustrate how XS EC’s explanations can be inspected by an analyst. We focus on network intrusion detection because its flow-level features are relatively interpretable, but the same analysis can be applied to the other datasets. Our goal is to show that XS EC highlights features that are consistent with class-discriminative patterns in the data. In the network intrusion dataset, we find that minimum time to live (MIN_TTL) is a strong indicator of whether a flow is benign or malicious. This observation is dataset-specific: MIN_TTL is not universally indicative of malicious traffic, but in this dataset its distribution differs substantially between benign and attack flows. Most benign flows have MIN_TTL= 31, whereas malicious flows often have different, typically higher, values. Since TTL values reflect network path and hostconfiguration characteristics, this feature helps distinguish benign and attack traffic in this dataset. Thus, for a benign flow with low MIN_TTL, we expect XS EC to classify the sample

Table 7: A case study of the network intrusion dataset. The table shows examples along with their ground truth (GT), the value of MIN_TTL, prediction (Pred), and the top-ranked important features identified by XS EC. The most important features are ordered by row–the first row contains features with the highest scores. GT

MIN_TTL

Benign

Malicious

254

Pred

Explanation DNS_QUERY_TYPE= 1, TCP_FLAGS= 0, MIN_TTL= 31.0 Benign MIN_IP_PKT_LEN= 73 DNS_TTL_ANSWER= 60 NUM_PKTS_128_TO_256_BYTES= 0, IN_PKTS= 2 Malicious MIN_TTL= 254 LONGEST_FLOW_PKT= 100

as benign and assign high importance to MIN_TTL; similarly, for a malicious flow with high MIN_TTL, we expect XS EC to identify MIN_TTL as evidence for the malicious prediction. Table 7 shows one true negative and one true positive example. In both cases, XS EC ranks MIN_TTL among the top-3 most important features. For the true negative sample, MIN_TTL= 31, and XS EC correctly predicts the flow as benign. For the true positive sample, MIN_TTL= 254, and XS EC correctly predicts the flow as malicious. These examples show that XS EC’s explanations align with a feature that separates benign and malicious flows in the dataset. We also compare these explanations with LIME and SHAP using an MLP target model that correctly classifies the same two samples. For the benign sample, LIME ranks MIN_TTL as the 10th most important feature and SHAP ranks it 8th. For the malicious sample, LIME ranks MIN_TTL 31st, and SHAP ranks it 10th. In contrast, XS EC ranks MIN_TTL in the top 3 for both samples, making the class-discriminative feature more prominent in the explanation.

Limitations and Discussion

Robustness to Adversarial Perturbations. In the context of XAI, adversarial perturbations aim to change (1) the model prediction, or (2) the generated explanations while retaining the predicted label [8, 50]. Formally, if f is the classifier and e is the XAI method that operates on f and a data point x,

Table 8: Results of robustness analysis of XS EC and ProtoPNet. FGSM and PGD aim to change the predicted label with varying levels of adversarial budget ε. Each cell displays the attack success rate (ASR) with the corresponding setup.

XS EC ProtoPNet

ε = 0.01 5.75% 6.55%

FGSM ε = 0.1 14.45% 28.35%

ε = 0.3 46.95% 41.45%

ε = 0.01 5.75% 6.80%

PGD ε = 0.1 14.70% 38.15%

Table 9: Results of an existing attack [8] on XS EC and LIME. The attack attempts to change the explanation arbitrarily without affecting the prediction. XS EC LIME

ε = 0.3 52.15% 84.15%

the goal of such attacks is to find a perturbation δ such that f (x + δ) ̸= f (x) or e( f , x + δ) ̸= e( f , x). Prior work shows that XAI methods are vulnerable to whitebox attacks that can produce arbitrary explanations [8] or labels [67]. Recent work [50] has shown that post-hoc approximation methods, e.g., LIME and SHAP, are vulnerable to attacks that create arbitrary explanations that do not reflect the biases in the original model. We evaluate the robustness of XS EC in the presence of adversarial perturbations that aim to achieve each of the above objectives on the Website Phishing dataset. We leverage FGSM [13] and PGD [28] to evaluate the effect of perturbations on predictions of XS EC and ProtoPNet. The attack success rates are presented in Table 8. Both XS EC and ProtoPNet are affected similarly with increasing ASR as attack budget increases. We also modify an existing targeted attack [8] into an untargeted attack to study how adversarial perturbations affect the explanations generated by XS EC while preserving the predicted label. We evaluate the same attack on LIME and report the results in Table 9. We use two metrics: attack success rate (ASR), defined here as the proportion of samples whose predicted label remains unchanged after perturbation, and cosine similarity between the original explanation and the explanation for the perturbed sample. A higher ASR indicates that the attack successfully preserves the model prediction, while a lower cosine similarity indicates a larger change in the explanation. The attack preserves the predicted label for most samples for both methods. However, XS EC’s explanations change less than LIME’s under this attack, as reflected by the higher cosine similarity. These results suggest that XS EC provides better explanation stability than LIME under this specific perturbation setting. However, they do not establish general robustness against adaptive explanation attacks, which remains an important direction for future work. Failure Analysis. We examine two network intrusion examples where XS EC misclassifies the sample. These examples illustrate how XS EC’s explanations can help diagnose model errors by revealing when the prediction relies on features whose values are atypical or inconsistent with the expected class pattern. Table 10 shows one false negative and one false positive

Attack Success Rate Cosine Similarity 99.55% 0.754 100.0% 0.131

Table 10: Examples of failure cases of XS EC on the network intrusion dataset. GT

MIN_TTL

Malicious

Benign

254

Pred

Explanation DNS_QUERY_TYPE= 0, TCP_FLAGS= 0, MIN_TTL= 0 Benign RETRANSMITTED_OUT_BYTES= 0, DNS_TTL_ANSWER= 0 NUM_PKTS_128_TO_256_BYTES= 0, IN_PKTS= 2 Malicious MIN_TTL= 254, DST_TO_SRC_SECOND_BYTES= 0 LONGEST_FLOW_PKT= 44

example. In the false negative case, XS EC misclassifies a malicious flow as benign because it assigns high importance to the low value of MIN_TTL, which is typically associated with benign traffic in this dataset. Further inspection shows that this sample is anomalous, with most feature values equal to 0. In the false positive case, XS EC predicts a benign flow as malicious because MIN_TTL= 254, a value more commonly associated with malicious flows in our case study (see Section 5.7). These examples show that XS EC’s explanations can help analysts identify when errors arise from atypical feature patterns rather than opaque model behavior. Guidelines for Training. The goal when training XS EC is to balance predictive performance with explanation quality. A self-explainable model with poor classification performance is not useful, even if its explanations are sparse or stable; conversely, an accurate model may not produce the most faithful or interpretable explanations. In practice, we first tune the architecture, learning rate, batch size, embedding dimension, and number of prototypes per class to achieve strong validation accuracy. We then tune the loss coefficients to improve explanation fidelity and sparsity while maintaining comparable classification performance. The explanation-specific losses should be adjusted jointly. Increasing the sparsity loss encourages concise explanations, but overly strong sparsity can remove informative features or lead to degenerate masks. The binary-mask loss should be large enough to produce interpretable near-binary masks, while the mask-similarity loss helps prevent different prototypes from collapsing to the same feature subset. The cluster loss should be tuned to maintain meaningful prototype alignment without sacrificing classification performance. We therefore select the final model based on validation accuracy, fidelity, and sparsity, rather than accuracy alone.

Conclusion

We introduce XS EC, a prototype-based self-XAI architecture that jointly performs classification and generates featureimportance scores as explanations. XS EC learns class-specific prototypes and prototype-specific masks, using prototype similarities for prediction and explanation. Across five security datasets, XS EC achieves competitive predictive performance while producing explanations that are faithful, sparse and perfectly stable. These results show that XS EC is a significant advancement towards interpretable security models.

Acknowledgments This material is based upon work supported by the National Science Foundation (NSF) under grant no. 2229876 and is supported in part by funds provided by the NSF, by the Department of Homeland Security, and by IBM. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF or its federal agency and industry partners.

References [1] Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey E. Hinton. Neural additive models: interpretable machine learning with neural nets. In Neural Information Processing Systems, 2021. [2] David Alvarez-Melis and Tommi S. Jaakkola. Towards robust interpretability with self-explaining neural networks. In Advances in Neural Information Processing Systems, 2018. [3] Sercan Ö Arik and Tomas Pfister. Tabnet: Attentive interpretable tabular learning. In AAAI Conference on Artificial Intelligence, 2021. [4] Dipkamal Bhusal, Rosalyn Shin, Ajay Ashok Shewale, Monish Kumar Manikya Veerabhadran, Michael Clifford, Sara Rampazzi, and Nidhi Rastogi. Sok: Modeling explainability in security analytics for interpretability, trustworthiness, and usability. In International Conference on Availability, Reliability and Security, 2023. [5] John S. Bridle. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In Neurocomputing, 1990. [6] Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, et al. This looks like that: Deep learning for interpretable image recognition. In Advances in Neural Information Processing Systems, 2019.

[7] Kang Leng Chiew, Choon Lin Tan, KokSheik Wong, Kelvin S.C. Yong, and Wei King Tiong. A new hybrid ensemble feature selection framework for machine learning-based phishing detection system. Information Sciences, 2019. [8] Ann-Kathrin Dombrowski, Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus-Robert Müller, and Pan Kessel. Explanations can be manipulated and geometry is to blame. In Advances in Neural Information Processing Systems, 2019. [9] Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, et al. Concept embedding models: Beyond the accuracy-explainability trade-off. Advances in Neural Information Processing Systems, 2022. [10] Mateo Espinosa Zarlenga, Zohreh Shams, Michael Edward Nelson, Been Kim, and Mateja Jamnik. Tabcbm: Concept-based interpretable neural networks for tabular data. Transactions on Machine Learning Research, 2023. [11] Kevin Fauvel, Fuxing Chen, and Dario Rossi. A lightweight, efficient and explainable-by-design convolutional neural network for internet traffic classification. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023. [12] Ruth C. Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. In IEEE International Conference on Computer Vision, 2017. [13] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In arXiv preprint arXiv:1412.6572, 2014. [14] Bryce Goodman and Seth Flaxman. European union regulations on algorithmic decision making and a “right to explanation”. In AI Magazine, 2017. [15] Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J. Cree. Regularisation of neural networks by enforcing lipschitz continuity. In Machine Learning, 2021. [16] Wenbo Guo, Dongliang Mu, Jun Xu, Purui Su, Gang Wang, and Xinyu Xing. Lemna: Explaining deep learning based security applications. In ACM SIGSAC Conference on Computer and Communications Security, 2018. [17] Wenbo Guo, Dongliang Mu, Jun Xu, Purui Su, Gang Wang, and Xinyu Xing. Lemna: Explaining deep learning based security applications. https://github.

com/Henrygwb/Explaining-DL, 2018. 2025-04-08.

Accessed:

[18] Qihan Huang, Mengqi Xue, Wenqi Huang, Haofei Zhang, Jie Song, Yongcheng Jing, and Mingli Song. Evaluation and improvement of interpretability for selfexplainable part-prototype networks. In IEEE/CVF International Conference on Computer Vision, 2023. [19] Jerzy Stefanowski Jacek Karolczak. An interpretable prototype parts-based neural network for medical tabular data. In Second Workshop on Explainable Artificial Intelligence for the Medical Domain, 2025. [20] Xiangjian Jiang, Andrei Margeloiu, Nikola Simidjievski, and Mateja Jamnik. Protogate: Prototype-based neural networks with global-to-local feature selection for tabular biomedical data. In International Conference on Machine Learning, 2024. [21] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. [22] Rune Kjærsgaard, Ahcene Boubekki, and Line Clemmensen. Pantypes: Diverse representatives for selfexplainable models. In AAAI Conference on Artificial Intelligence, 2024. [23] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, et al. Concept bottleneck models. In International Conference on Machine Learning, 2020. [24] Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, R. Howard, Wayne Hubbard, and Lawrence Jackel. Handwritten digit recognition with a backpropagation network. In Neural Information Processing Systems, 1989. [25] Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In AAAI Conference on Artificial Intelligence, 2018. [26] Scott Lundberg and Su-In Lee. Shap: Shapley additive explanations. https://github.com/shap/shap, 2017. Accessed: 2025-04-08.

[29] Dang Minh, H Xiang Wang, Y Fen Li, and Tan N Nguyen. Explainable artificial intelligence: a comprehensive review. Artificial Intelligence Review, 2022. [30] Jaron Mink, Harjot Kaur, Juliane Schmüser, Sascha Fahl, and Yasemin Acar. Security is not my field, i’m a stats guy: A qualitative root cause analysis of barriers to adversarial machine learning defenses in industry. In USENIX Security Symposium, 2023. [31] Nour Moustafa, Gideon Creech, and Jill Slay. Big data analytics for intrusion detection system: Statistical decision-making using finite dirichlet mixture models. In Data Analytics and Decision Support for Cybersecurity: Trends, Methodologies and Applications, 2017. [32] Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive dataset for network intrusion detection systems (unsw-nb15 network dataset). In Military Communications and Information Systems Conference, 2015. [33] Nour Moustafa and Jill Slay. The evaluation of network anomaly detection systems: Statistical analysis of the unsw-nb15 dataset and the comparison with the kdd99 dataset. In Information Security Journal: A Global Perspective, 2016. [34] Nour Moustafa, Jill Slay, and Gideon Creech. Novel geometric area analysis technique for anomaly detection using trapezoidal area estimation on large-scale networks. In IEEE Transactions on Big Data, 2019. [35] Meike Nauta, Ron van Bree, and Christin Seifert. Neural prototype trees for interpretable fine-grained image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021. [36] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, et al. Pytorch: an imperative style, high-performance deep learning library. In International Conference on Neural Information Processing Systems, 2019. [37] Captum Source Repository. Captum: Model interpretability for pytorch. https://github.com/ pytorch/captum, 2019. Accessed: 2025-04-08.

[27] Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In International Conference on Neural Information Processing Systems, 2017.

[38] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Lime: Local interpretable model-agnostic explanations. https://github.com/marcotcr/lime, 2016. Accessed: 2025-04-08.

[28] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.

[39] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should i trust you?": Explaining the predictions of any classifier. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016.

[40] Dawid Rymarczyk, Lukasz Struski, Michal Górszczak, Koryna Lewandowska, Jacek Tabor, and Bartosz Zieliński. Interpretable image classification with differentiable prototypes assignment. In European Conference on Computer Vision, 2022.

[50] Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 2020.

[41] Dawid Rymarczyk, Lukasz Struski, Jacek Tabor, and Bartosz Zieliński. Protopshare: Prototypical parts sharing for similarity discovery in interpretable image classification. In ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021.

[51] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. In arXiv preprint arXiv:1706.03825, 2017.

[42] Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. Evaluating the visualization of what a deep neural network has learned. IEEE Transactions on Neural Networks and Learning Systems, 2017. [43] Francesco De Santis, Philippe Bich, Gabriele Ciravegna, Pietro Barbiero, Danilo Giordano, and Tania Cerquitelli. Self-supervised interpretable concept-based models for text classification. In arXiv preprint arXiv:2406.14335, 2024. [44] Mohanad Sarhan, Siamak Layeghy, Nour Moustafa, and Marius Portmann. Netflow datasets for machine learning-based network intrusion detection systems. In Big Data Technologies and Applications, 2021. [45] Mohanad Sarhan, Siamak Layeghy, and Marius Portmann. Towards a standard feature set for network intrusion detection system datasets. In Mobile Networks and Applications, 2022. [46] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, et al. Gradcam: Visual explanations from deep networks via gradient-based localization. In International Conference on Computer Vision, 2017. [47] Ryan Sheatsley, Blaine Hoak, Eric Pauley, Yohan Beugin, Michael Weisman, and Patrick McDaniel. On the robustness of domain constraints. In ACM SIGSAC Conference on Computer and Communications Security, 2021. [48] Jacob Si, Wendy Yusi Cheng, Michael Cooper, and Rahul G. Krishnan. Interpretabnet: distilling predictive signals from tabular data by salient feature interpretation. In International Conference on Machine Learning, 2024. [49] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In International Conference on Learning Representations, 2014.

[52] Charles Smutz and Angelos Stavrou. Malicious pdf detection using metadata and structural features. In Annual Computer Security Applications Conference, 2012. [53] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. In International Conference on Learning Representations, 2015. [54] Nedim Srndic and Pavel Laskov. Practical evasion of a learning-based classifier: A case study. In IEEE Symposium on Security and Privacy, 2014. [55] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, 2017. [56] Mahbod Tavallaee, Ebrahim Bagheri, Wei Lu, and Ali A. Ghorbani. A detailed analysis of the kdd cup 99 data set. In IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009. [57] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017. [58] Jiaqi Wang, Huafeng Liu, Xinyue Wang, and Liping Jing. Interpretable image recognition by constructing transparent embedding space. In IEEE/CVF International Conference on Computer Vision, 2021. [59] Alexander Warnecke, Daniel Arp, Christian Wressnegger, and Konrad Rieck. Evaluating explanation methods for deep learning in security. In IEEE European Symposium on Security and Privacy (EuroS&P), 2020. [60] Feng Wei, Hongda Li, Ziming Zhao, and Hongxin Hu. xnids: Explaining deep learning-based network intrusion detection systems for active intrusion responses. In USENIX Security Symposium, 2023. [61] Feng Wei, Hongda Li, Ziming Zhao, and Hongxin Hu. xnids: Explaining deep learning-based network intrusion detection systems for active intrusion responses. https://github.com/CactiLab/code-xNIDS, 2023. Accessed: 2025-04-08.

[62] Limin Yang, Arridhana Ciptadi, Ihar Laziuk, Ali Ahmadzadeh, and Gang Wang. Bodmas: An open dataset for learning based temporal analysis of pe malware. In 4th Deep Learning and Security Workshop, 2021. [63] Limin Yang, Wenbo Guo, Qingying Hao, Arridhana Ciptadi, Ali Ahmadzadeh, et al. Cade: Detecting and explaining concept drift samples for security applications. In USENIX Security Symposium, 2021. [64] Renos Zabounidis, Joseph Campbell, Simon Stepputtis, Dana Hughes, and Katia P. Sycara. Concept learning for interpretable multi-agent reinforcement learning. In Conference on Robot Learning, 2023. [65] Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, et al. Concept embedding models: Beyond the accuracyexplainability trade-off. In Advances in Neural Information Processing Systems, 2022. [66] Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision, 2014. [67] Xinyang Zhang, Ningfei Wang, Hua Shen, Shouling Ji, Xiapu Luo, and Ting Wang. Interpretable deep learning under fire. In USENIX Security Symposium, 2020. [68] Luisa M. Zintgraf, Taco S. Cohen, Tameem Adel, and Max Welling. Visualizing deep neural network decisions: Prediction difference analysis. In International Conference on Learning Representations, 2017.

Details of Datasets

Here we present details of the datasets we use for evaluation. PDF Malware Identification. This dataset includes features (e.g., number of characters in the title) extracted from PDF files labeled malware or benign [52, 54]. We use 135 features extracted from ∼ 5K benign and 5K malicious (malware) PDF files, following previous work [16]. Phishing Website Detection. We use 38 features (e.g., URL length, presence of specific keywords associated with phishing) from a phishing dataset [7] that contains features of a website labeled either as a phishing website or a benign website. 5K phishing web page URLs are collected from PhishTank and OpenPhish, and 5K legitimate web page URLs are from Alexa and the Common Crawl5 archive. Network Intrusion Detection. We use 39 flow-based features (e.g., packet counts, byte counts, flow duration) from the Netflow dataset [45], an extension of the popular dataset UNSW-NB15 [31–34, 44]. We randomly sample 10K benign and 10K attack samples for our experiments.

Table 11: Hyper-parameters and network architecture details obtained by training XS EC on each dataset. Λ is the vector of coefficients in the same order as Equation 10. Learning # Neurons # Neurons Embedding Batch k Λ Rate in M in f Dimension Size PDF 512, 1024, 1.0, 3.5, 0.8, 1e−3 256, 128, 64 128 10 256 Malware 2048 0.5, 0.5 Website 256, 512, 3.5, 5.0, 0.5, 5e−3 256, 128, 64 128 10 256 Phishing 1024 0.5, 0.5 Network 512, 1024, 1.0, 5.0, 0.8, 1e−3 256, 128, 64 128 10 256 Intrusion 2048 0.5, 0.5 PE 256, 512, 1.0, 15.0, 0.8, 1e−3 256, 128, 64 128 10 256 Malware 1024 0.5, 0.5 Network 256, 512, 1.0, 15.0, 0.8, 1e−3 4 4 10 256 Attack 1024 0.5, 0.5 Dataset

PE Malware Classification. The BODMAS dataset [62] includes 2,381 features on benign and 581 malware families extracted from program executables. We use all samples from malware families that have at least 3,000 samples, which are 5 classes. We then remove constant features and select 100 features with the highest ANOVA F-value scores for our experiments. Network Attack Classification. The NSL-KDD dataset [56] is a widely used dataset that has flow-based features with temporal relationships. The labels are either benign or the network attack type. We preprocess the dataset and select all attacks that have at least 3,000 samples, i.e., neptune, satan, ipsweep, and smurf attacks.

Hyperparameters of XS EC

The hyperparameters of XS EC’s training on each dataset are given in Table 11. The feature encoder is an MLP for the PDF malware, phishing website, network intrusion, and PE malware datasets and an LSTM for the network attack dataset. This is because the network attack dataset has temporal relationships between features, which are better modeled by an LSTM.

Record · ID 432468 · SHA-256 97ad9b7eee852e1e
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.