Between Zeros and Ones: Behavioral Characterization Beyond Binary Labeling Across Public ICS Datasets
arXiv:2606.30493v1 [cs.CR] 29 Jun 2026
Konstantinos E. Kampourakis1[0009−0000−8883−0735] , Vyron Kampourakis1[0000−0003−4492−5104] , Georgios Spathoulas1[0000−0003−2947−486X] , and Constantinos Kolias2[0000−0002−3020−291X] 1
Norwegian University of Science and Technology, 2802 Gjøvik, Norway {konstantinos.kampourakis, vyron.kampourakis, georgios.spathoulas}@ntnu.no 2 University of Idaho, Idaho Falls, ID 83402, USA [email protected]
Abstract. Intrusion detection in Industrial Control Systems (ICS) is typically evaluated on a small set of public benchmarks using binary “normal” versus “attack” labels, a practice that can mask the behavioral diversity of cyber-physical attacks. To address this limitation, we propose a behavioral characterization framework that maps raw multivariate process traces into five interpretable physical primitives: drift, spike, oscillation, repetition, and switching. We apply the framework to three widely used ICS benchmarks, namely, SWaT, WADI, and HAI, and show that attack windows exhibit clear behavioral shifts relative to normal operation while the three datasets occupy largely distinct regions of the behavioral space, revealing both cross-dataset bias and intra-dataset diversity. In particular, WADI is dominated by repetition, HAI emphasizes sustained drift and oscillation, and SWaT is characterized by stealthier frozen-telemetry behavior. To examine the evaluation implications, we use an indicative Random Forest baseline and show that aggregate binary metrics can limit visibility into performance across different behavioral proxies. For example, in SWaT, macro F1 drops from 85.44% under binary evaluation to 37.84% under behavior-proxy multiclass prediction, with similar degradations observed on WADI and HAI. Based on these findings, we argue for complementing conventional binary benchmarking with behavior-stratified evaluation to expose blind spots that aggregate scores leave hidden and to better support targeted incident response. Keywords: Industrial Control Systems · Intrusion Detection · Behavioral Characterization · Benchmark Bias · Cyber-Physical Security · Dataset Analysis.
1
Introduction
Public ICS datasets such as SWaT [16], WADI [2], and HAI [22] have become de facto benchmarks for evaluating Intrusion Detection Systems (IDS) targeting
2
V. Kampourakis et al.
cyber-physical attacks. Most studies treat these datasets as labeled time series in which each sample receives a binary attack-or-normal label [3, 5, 8, 17], and evaluate IDS models using aggregate metrics such as precision, recall, and F1score. This binary labeling practice implicitly assumes that the attack class is homogeneous. In reality, industrial processes can be manipulated in many ways, including gradual sensor drifts, abrupt spikes, frozen telemetry, oscillatory control behavior, and forced actuator switching [9, 18, 21, 27]. These behaviors are visible in the temporal structure of process traces but are typically not distinguished in IDS evaluation. However, ignoring behavioral diversity can have practical consequences. Unlike traditional IT security, where the response to an intrusion is largely uniform, ICS environments require differentiated operator responses. For example, a spike indicating a valve slam may require an immediate safety trip, whereas a slow sensor drift may call for recalibration over several hours. In this context, binary detection collapses these qualitatively different urgencies into a single signal, leaving operators without the behavioral specificity needed to respond appropriately. To address this, we introduce a behavioral characterization framework that maps multivariate ICS traces into five interpretable temporal primitives: drift, spike, oscillation, repetition, and switching. The framework is purely observational and exploratory; namely, it quantifies how attacks may manifest in normalized process traces, but does not infer attacker intent, semantic attack types, or a universal attack taxonomy. This scope is intentional; rather than requiring process-specific domain knowledge or behavioral ground truth, the framework characterizes what is directly observable and reproducible across datasets. This makes it a portable first step toward exposing the behavioral diversity that binary labels may leave unmeasured. We apply it to SWaT, WADI, and HAI and address four research questions: – RQ1: Do attack windows exhibit behavioral characteristics different from normal operation? – RQ2: Do the datasets differ in the distribution of normalized behavioral morphologies captured by our primitives? – RQ3: Are attack events internally diverse in their behavioral structure? – RQ4: How does binary IDS evaluation compare with proxy-based evaluation in distinguishing attack-window behaviors? The rest of the paper is structured as follows. Section 2 reviews related work. Section 3 presents the framework. Section 4 reports empirical findings. Sections 5 and 6 discuss takeaways and limitations. The last section concludes and provides pointers for future work.
2
Related Work
This section briefly discusses related work on ICS security datasets, intrusiondetection evaluation practices, and benchmark limitations relevant to our study.
Title Suppressed Due to Excessive Length
3
Public ICS benchmarks such as SWaT [16], WADI [2], and HAI [22] are widely used for data-driven IDS research. Although additional cyber-physical security datasets have also been proposed [13,19,23], evaluation remains concentrated on a small set of public benchmarks [5, 12]. Existing comparisons usually organize datasets by domain, architecture, or protocol rather than by the behavioral characteristics of the process traces they contain [12]. In this work, we focus on SWaT, WADI, and HAI because they are public, process-centric, and broadly adopted benchmarks for multivariate time-series anomaly detection [3, 7]. Intrusion detection in ICS has been studied using physics-based methods, ML models, and more recently, LLM-assisted schemes [1,6,14,21,24,25]. Despite the methodological differences, evaluation is commonly formulated as binary classification with aggregate metrics such as precision, recall, and F1-score [3,7]. This setup implicitly treats attacks as a single class, even though cyber-physical attacks may induce qualitatively different process effects, including gradual drift, abrupt transients, oscillatory behavior, stale telemetry, or excessive actuator switching [11,18,27]. Our work departs from this detector-centric perspective by characterizing attack windows through interpretable behavioral properties and using them to stratify evaluation. Prior studies have already noted that ICS benchmarks capture only specific testbeds and attack scenarios; therefore, supporting limited generalization [16, 22, 23]. Systematization of Knowledge (SoK) papers and surveys further show that IDS evaluations are often biased by dataset choice and by overreliance on aggregate metrics [6, 12]. However, existing analyses do not formalize processbehavioral bias, that is, which temporal attack behaviors are over- or underrepresented in standard benchmarks. To address this gap, we quantify behavioral coverage using five interpretable temporal properties and examine how SWaT, WADI, and HAI populate this space differently.
3
Methodology
The proposed methodology characterizes cyber-physical attacks through their observable temporal effects on industrial process dynamics, without, however, modeling attacker intent or process-specific semantics. Specifically, the framework transforms raw multivariate ICS traces into behavioral embeddings that capture generic operational disturbances across heterogeneous industrial environments. Figure 1 provides an overview of the framework’s pipeline, in which raw process traces are segmented into overlapping temporal windows, from which lightweight behavioral primitives are extracted and aggregated into low-dimensional behavioral vectors. Overall, the framework consists of four stages: preprocessing, sliding-window segmentation, behavioral primitive extraction, and behavioral embedding generation. 3.1
Preprocessing and Windowing
The utilized ICS datasets, i.e., SWaT, WADI, and HAI, consist of multivariate temporal traces containing both continuous process variables (e.g., tank levels,
4
V. Kampourakis et al.
Fig. 1: Overview of the proposed behavioral characterization framework.
flow rates, pressures) and discrete operational states (e.g., pump activity, valve positions). Let the process state at time step t consist of N sensor and actuator variables. To ensure that variables with substantially different physical units contribute equitably to the behavioral analysis, all continuous process variables are normalized using z-score normalization, as seen in equation 1. Note that we choose z-normalization because it is simple, widely used, and provides a common scale that preserves relative deviations within each variable. The normalization statistics are computed from normal operating data to prevent attack periods from contaminating the baseline process distribution. For a continuous variable x (e.g., a sensor time series), normalization is performed as: x − µnormal (1) σnormal + ϵ where µnormal and σnormal denote the mean and standard deviation estimated from normal operation only, and ϵ is a small constant added for numerical stability. The latter is added in the denominator to avoid division by zero for variables with near-zero variance, without materially affecting the resulting z-scores. Discrete actuator states are intentionally left unnormalized to preserve their categorical operational semantics. Following preprocessing, the multivariate traces are segmented into overlapping temporal windows. Specifically, the proposed framework utilizes a fixed window length of w = 60 samples with a 50% overlap between adjacent windows. Regarding the latter, the 50% overlap was used to reduce the effects of boundary fragmentation and preserve the temporal continuity between adjacent windows. Simply put, without overlap, short-duration or transitional behaviors may be split across neighboring z=
Title Suppressed Due to Excessive Length
5
windows, weakening their representation. On the other hand, larger overlaps may increase computational cost and redundancy, while providing limited additional temporal information. Therefore, the empirically selected overlap represents a compromise between temporal coverage and computational efficiency. To further justify the selection, specifically for the window length, a sensitivity analysis is provided in Appendix A. Utterly, each window receives a binary label based on the proportion of attack-labeled samples contained within the window. A window is assigned an attack label (yw = 1) if at least 50% of its constituent samples correspond to attack periods in the ground truth of the dataset. Otherwise, the window is labeled as normal (yw = 0). The readers should keep in mind that the threshold selection is a practical choice rather than a universal solution; different window sizes and overlap ratios could be used, depending on the temporal resolution and dynamics of the dataset.
3.2
Behavioral Primitive Design
The proposed framework employs a set of interpretable behavioral primitives derived from classical time-series statistics and does not rely on learned latent representations or process-specific feature engineering. The primitives are intentionally lightweight and computationally inexpensive, enabling applicability across heterogeneous ICS datasets without assumptions regarding process physics or control architecture. In other words, based on common empirical observations during adversarial activity in CPS [11,25,27], primitives were selected according to three criteria: (i) interpretability, (ii) computational simplicity, and (iii) crossdataset applicability. Of course, the selected primitives should be viewed as a conservative baseline rather than an exhaustive characterization library. Alternative descriptors, including entropy-based measures, spectral features, wavelets, lag-response statistics, and nonlinear dynamical metrics, are plausible candidates for future primitive-selection studies.
Table 1: Behavioral interpretation of the proposed primitives. Primitive Captured Behavior Drift
Sustained monotonic deviation
Physical Manifestation
Sensor bias injection, gradual process manipulation, slow tank level deviation Spike Abrupt transient change Sudden valve closure/opening, pressure surge, transient disturbance Oscillation Repeated directional reversals Unstable control loops, actuator hunting, feedback instability Repetition Recurring temporal patterns Replay attacks, periodic reuse of process trajectories Switching Excessive discrete state transitions Forced actuator toggling, abnormal valve or pump switching
6
V. Kampourakis et al.
Table 1 summarizes the operational interpretation of the behavioral primitives proposed. The selected primitives are chosen as a compact baseline that spans five broad and interpretable disturbance families commonly observable in process traces: sustained deviation (drift), abrupt transient change (spike), directional instability (oscillation), recurring temporal structure (repetition), and excessive discrete switching (switching), as seen in the table. 3.3
Behavioral Primitive Extraction
For each temporal window and for each normalized continuous signal x, the behavioral primitives introduced in section 3.2 are computed. For notational simplicity, the first-order temporal difference is denoted as ∆xt = xt − xt−1 . Importantly, the formulas rely only on elementary time-series operations, including variance, linear slope, maximum derivative, sign changes, and autocorrelation, in line with standard anomaly detection practice to provide strong baselines [10, 20, 26] with simple statistical features. Drift Intensity (D). Drift intensity captures sustained monotonic deviations and gradual process shifts. To quantify persistent trends, a linear model xt = mt+b is fitted within the window. The drift score combines trend magnitude with regression fit quality, as shown in equation 2. Note that m denotes the fitted slope and R2 denotes the coefficient of determination. Higher values indicate stronger, persistent directional behavior. D = |m| · R2
(2)
Spike Intensity (S). Spike intensity captures abrupt transient disturbances and sudden process deviations. This is approximated using the maximum absolute first-order temporal difference, as shown in equation 3. Higher values indicate sharper transient process changes. S = max |∆xt | t
(3)
Oscillation Intensity (O). Oscillation intensity captures unstable control behavior and repeated directional reversals in process dynamics. The metric is computed as the normalized frequency of sign changes in the first derivative, as shown in equation 4. Higher values indicate stronger oscillatory behavior. O=
w 1 X 1 sgn(∆xt ) ̸= sgn(∆xt−1 ) w t=2
(4)
Repetition Intensity (R). Repetition intensity captures recurring temporal structures and replay-like signal behavior. The metric is computed using the autocorrelation function ρ(ℓ) and extracting the maximum correlation over a bounded lag interval, as shown in equation 5. The restricted lag interval avoids
Title Suppressed Due to Excessive Length
7
trivial zero-lag correlations while focusing on short-term recurring temporal structure. R = max ρ(ℓ)
(5)
ℓ∈[5,30]
Switching Intensity (A). For discrete actuator signals ut ∈ {0, 1, . . . , k}, continuous statistical metrics are physically inappropriate. Instead, switching intensity quantifies actuator toggling and discrete operational state transitions by counting the number of state changes within the window, as shown in equation 6. Higher values indicate more frequent operational switching activity. A=
w X
1[ut ̸= ut−1 ]
(6)
t=2
Importantly, the afore-described definitions of the primitives should be understood as operational proxies for observable temporal behavior rather than complete physical definitions. For example, the drift metric captures sustained monotonic deviation but not all nonlinear drift forms; the spike metric captures abrupt transients but may not fully distinguish a sharp step followed by a plateau; the oscillation metric captures repeated directional reversals but not every limit-cycle regime; and the repetition metric captures short-lag recurrence but may fail to represent all replay-like or phase-shifted behaviors. Likewise, repetition should not be interpreted as a standalone anomaly signal, since industrial processes are often intrinsically repetitive during normal operation; its value lies primarily in combination with the other primitives when characterizing how attack windows differ behaviorally from nominal process progression. 3.4
Feature Vector Construction
Cyber-physical attacks typically target only a small subset of process variables. Because of this, simply averaging behavioral scores across all variables would eliminate the localized effects of an attack. Conversely, taking the absolute maximum score across all variables could render the framework sensitive to isolated noise spikes. To strike a balance, we aggregate the primitive scores (drift, spike, oscillation, repetition, and switching) across all N process variables using the 95th percentile operator. Conceptually, this means sorting all datasets’ variables by how strongly they exhibit a given behavior, discarding the top 5% to filter out isolated sensor glitches, and retaining the highest remaining value. Mathematically, this corresponds to the threshold below which 95% of the variable scores fall, as shown in equation 7. Recall that the percentile is computed spatially (across the N variables) and not temporally (across the entire dataset). This means that the embedding is generated on a per-window basis and enables characterization even in an online, real-time IDS evaluation. P95 = Q(0.95),
(7)
8
V. Kampourakis et al.
Note that Q(·) denotes the empirical quantile function, which calculates percentiles directly from the datasets. This approach highlights localized attack behaviors while filtering out extreme, single-variable outliers. For each temporal window, we apply this aggregation to all five primitives to construct a compact, five-dimensional feature vector, as shown in equation 8. This vector serves as the foundational representation for all subsequent statistical analyses, including the behavioral coverage evaluations in sections 4.2 and 4.3. vw = [D95 , S95 , O95 , R95 , A95 ] ∈ R5 . 3.5
(8)
Dominant Behavioral Property Assignment
For coarse behavior-stratified analysis and visualization, we assign each temporal window a single dominant behavioral property. Note that this label is a heuristic summary, not a claim that the window contains only one behavior. In practice, multiple primitives may co-occur within the same window, and some primitives may exhibit low baseline variance. Therefore, the dominant label should not be interpreted as a unique or exhaustive behavioral class. As explained in section 3.4, for each window, the behavioral embedding vector vw = [D95 , S95 , O95 , R95 , A95 ] collects the 95th-percentile activations of the five primitives across all process variables. Recall from section 3.1 that before dominant-property assignment, the five primitive scores are normalized using zscore normalization (equation 1) over the normal operating data of each dataset. The resulting normalized values are then compared to determine the dominant primitive. If the maximum normalized primitive activation falls below a threshold τ = 0.30 (see Appendix A), the window is labeled as unassigned, indicating the absence of a clearly dominant behavioral signature. Otherwise, the primitive corresponding to max(vw ) is assigned as the dominant property. The threshold is not intended to represent a universal decision boundary; instead, it filters out weak activations arising from ambient process noise or low-amplitude fluctuations. For transparency, note that a limitation of this approach is that collapsing each window to a single dominant label may obscure simultaneous behavior activations and does not preserve the full richness of the underlying process trace. 3.6
Behavioral Annotation Validation
The evaluated datasets provide only binary attack labels (attack vs. normal) and no behavior-level annotations; therefore, the proposed primitive assignments cannot be validated in a standard supervised manner against ground-truth behavioral classes. In this respect, to assess whether the automated labels correspond to recognizable patterns in the traces, we performed a human sanity check on 60 representative attack windows, sampled as 20 from SWaT, 20 from HAI, and 20 from WADI. Specifically, two annotators, namely the first two authors, visually inspected the multivariate trajectories and assigned each window a dominant observable behavior: drift, spike, oscillation, repetition, switching, or unassigned.
Title Suppressed Due to Excessive Length
9
Be aware that the annotations were made independently and without access to the automated framework outputs. κ=
po − pe 1 − pe
(9)
Agreement between the framework’s automated dominant label and the human annotation was measured using Cohen’s κ in Eq. 9. The value po denotes observed agreement and pe the chance agreement. The comparison yielded 98.3% observed agreement and a κ value of 0.978, which indicates high consistency between the automated assignments and the human annotations. Only one disagreement occurred among the 60 windows, in a WADI instance where one human annotator identified repetitive activity, the other identified an oscillatory pattern, and the framework assigned a spike label. This disagreement reflects the ambiguity that can arise when recurring patterns contain both oscillatory behavior and abrupt transient fluctuations. Because the datasets provide no behaviorlevel ground truth and the annotators were also the framework’s designers, the sanity check should be interpreted as a limited rather than an independent validation. Nevertheless, in general terms, the results suggest that the automated primitive assignments correspond to recognizable temporal patterns in the underlying process traces. Figure 2 presents representative examples from SWaT, HAI, and WADI, including the single disagreement case.
Fig. 2: Representative examples used during manual behavioral annotation.
10
4
V. Kampourakis et al.
Results
We evaluate the behavioral characterization framework proposed across the SWaT, WADI, and HAI datasets to answer the four research questions introduced in Section 1. Collectively, the results demonstrate that ICS attack traces exhibit substantial behavioral heterogeneity both across and within datasets. That is, attack windows occupy distinct regions of the proposed behavioral space depending on the temporal process dynamics induced during anomalous operation, and do not form a single homogeneous anomaly class. 4.1
RQ1: Behavioral Shift Between Normal and Attack States
To determine whether attack windows exhibit measurable behavioral deviations from normal operation, we first compare the distributions of the proposed behavioral primitives across normal and attack windows for the evaluated datasets, namely SWaT, WADI, and HAI. Figure 3 provides a visual overview, while Table 2 reports median and Interquartile Range (IQR) [4] values together with Mann–Whitney U [15] test results. Across the three datasets, attack windows differ measurably from normal windows, but the pattern of change depends on the primitive. As observed in Table 2, HAI has the clearest attack-associated increase in both drift and spike, with drift rising from 0.023 to 0.059 and spike from 1.732 to 2.150. SWaT shows a different profile, with oscillation increasing from 0.363 to 0.454 and repetition decreasing from 0.906 to 0.795. WADI exhibits smaller shifts overall, with spike increasing from 0.116 to 0.140, while oscillation remains essentially unchanged at 0.119 in both states. These examples indicate that attack behavior is datasetdependent rather than uniform across benchmarks. Collectively, table 2 confirms that most normal-versus-attack differences are statistically significant, showing also that statistical significance and effect magnitude are not the same thing. For example, WADI oscillation is not significantly different between normal and attack windows (p = 0.732), and WADI repetition changes only slightly in median terms despite a small p-value. Overall, the results indicate that attack windows consistently exhibit behavioral shifts relative to normal operation. Still, the dominant primitive depends on the dataset. Namely, HAI is driven more by spike and drift, SWaT by oscillation and reduced repetition, and WADI by smaller changes centered mainly on spike and drift. 4.2
RQ2: Dataset-Level Differences in Normalized Behavioral Morphology
To summarize the attack-window behavior in each dataset, we compute the median activation of each of the five behavioral primitives across all attack windows. Figure 4 visualizes these summaries, while Table 3 reports the corresponding median and IQR values. Recall that the primitives are computed from per-variable normalized signals (Section 3.1) and then aggregated across variables using the
Title Suppressed Due to Excessive Length
(a) Drift intensity
(b) Spike intensity
(c) Oscillation intensity
(d) Switching intensity
11
(e) Repetition intensity
Fig. 3: Distribution of behavioral primitive values for normal (0) and attack (1) windows across the evaluated datasets.
12
V. Kampourakis et al.
Table 2: Distributional comparison between normal and attack windows. Dataset Primitive Normal Median [IQR] Attack Median [IQR] p-value SWaT SWaT SWaT SWaT SWaT
Drift Spike Oscillation Repetition Switching
0.003 [0.002, 0.004] 0.216 [0.171, 0.275] 0.363 [0.308, 0.407] 0.906 [0.877, 0.937] 0.000 [0.000, 0.000]
0.001 [0.000, 0.003] 0.032 [0.019, 0.198] 0.454 [0.363, 0.498] 0.795 [0.707, 0.904] 0.000 [0.000, 0.000]
< 10−10 < 10−10 < 10−10 < 10−10 < 10−10
HAI HAI HAI HAI HAI
Drift Spike Oscillation Repetition Switching
0.023 [0.014, 0.035] 1.732 [1.412, 2.049] 0.644 [0.627, 0.665] 0.945 [0.938, 0.950] 0.000 [0.000, 0.000]
0.059 [0.034, 0.174] 2.150 [1.761, 2.539] 0.631 [0.610, 0.653] 0.948 [0.942, 0.952] 0.000 [0.000, 0.000]
< 10−10 < 10−10 < 10−10 < 10−6 < 10−10
WADI WADI WADI WADI WADI
Drift Spike Oscillation Repetition Switching
0.004 [0.002, 0.007] 0.116 [0.077, 0.200] 0.119 [0.119, 0.131] 0.938 [0.884, 0.945] 0.000 [0.000, 0.000]
0.005 [0.003, 0.008] < 10−10 0.140 [0.085, 0.225] < 10−4 0.119 [0.119, 0.136] 0.732 0.938 [0.903, 0.945] 0.010 0.000 [0.000, 0.009] < 10−10
95th percentile (Section 3.4); thus, they should be interpreted as unitless descriptions of temporal morphology rather than as directly comparable measures of physical severity across datasets.
Fig. 4: Heatmap of the mean normalized behavioral primitive activations across attack windows, after 95th-percentile aggregation over process variables.
Table 3 shows that the three datasets occupy different regions of the normalized behavioral space. HAI is the most spike-heavy, with a median spike of 2.150,
Title Suppressed Due to Excessive Length
13
Table 3: Attack-window behavioral coverage summarized as median [IQR] for each dataset and primitive. Dataset Primitive
Median [IQR]
SWaT SWaT SWaT SWaT SWaT
Drift 0.001 [0.000–0.003] Spike 0.032 [0.019–0.198] Oscillation 0.454 [0.363–0.498] Repetition 0.795 [0.707–0.904] Switching 0.000 [0.000–0.000]
HAI HAI HAI HAI HAI
Drift 0.059 [0.034–0.174] Spike 2.150 [1.761–2.539] Oscillation 0.631 [0.610–0.653] Repetition 0.948 [0.942–0.952] Switching 0.000 [0.000–0.000]
WADI WADI WADI WADI WADI
Drift 0.005 [0.003–0.008] Spike 0.140 [0.086–0.226] Oscillation 0.119 [0.119–0.136] Repetition 0.938 [0.903–0.945] Switching 0.000 [0.000–0.009]
compared with 0.140 in WADI and 0.032 in SWaT. It also has the highest drift and oscillation values among the three datasets. SWaT places more emphasis on oscillation and repetition, with medians of 0.454 and 0.795, while WADI is dominated by repetition, with a median of 0.938 and comparatively low drift and oscillation. Switching remains near zero in all three datasets. Overall, the table indicates that benchmarking on a single dataset emphasizes a particular normalized behavioral profile and may underrepresent others. Because the datasets also differ in sampling rate, process characteristics, and temporal granularity, these summaries should not be interpreted as comparisons over equivalent physical time spans. Instead, they describe how each benchmark populates the same normalized behavioral vocabulary. In that sense, table 3 provides a compact numerical view of dataset bias in behavioral morphology, rather than a ranking of attack severity across datasets. 4.3
RQ3: Intra-Dataset Behavioral Diversity
Although Section 4.2 establishes that the datasets are behaviorally biased away from each other, we further investigate whether attack events are homogeneous within a single dataset. While the heatmap in Figure 4 captures the dominant average behavioral traits of each benchmark, the presence of non-zero scores across multiple primitives indicates that attacks within the same dataset can induce substantially different temporal process behaviors. This effect is particularly evident in HAI, where attack windows exhibit strong spike, oscillation, and drift activations simultaneously, suggesting the coexistence of multiple behavioral attack patterns. Even in datasets with a more dominant behavioral sig-
14
V. Kampourakis et al.
Table 4: Binary versus behavior-proxy multiclass classification performance across datasets using macro metrics (%). Binary
Multiclass
Dataset Precision F1-score Recall Precision F1-score Recall SWaT HAI WADI
96.26 84.82 94.89
85.44 76.67 85.21
79.47 71.71 77.33
47.11 58.89 48.23
37.84 58.01 42.92
36.06 62.80 38.67
nature, such as SWaT and WADI, attack windows are not behaviorally uniform and still span multiple temporal primitives. For instance, a single IDS evaluation split may contain both an abrupt transient disturbance and a gradual process deviation. Under standard evaluation protocols, however, both are collapsed into the same binary label (“attack”). The underlying variation in these physical manifestations demonstrates that binary labels may obscure a substantial degree of behavioral diversity, treating a spectrum of attack-induced process behaviors as if they represented a single operational state. 4.4
RQ4: Behavior-Proxy Separability in Binary IDS Evaluation
Furthermore, to assess how well a baseline classifier separates attack windows according to our normalized behavioral proxies, we first report the standard binary setting and then provide a secondary heuristic multiclass reformulation using the dominant primitive labels defined in Section 3.2. Note that this multiclass result is descriptive rather than definitive, because it changes the task, the metric, and the target labels simultaneously. Specifically, each attack window was assigned the label of its dominant behavioral primitive, converting the problem from binary to multiclass classification. We emphasize that this is not a new IDS model or an exhaustive tuning exercise; rather, we use an indicative Random Forest classifier as a simple baseline for examining separability under the proposed proxy labels. Accordingly, the results in Table 4 should be interpreted as a descriptive test of within-dataset separability, not as evidence about physical attack taxonomy or operational severity. As shown in Table 4, binary classification yields higher macro scores than the multiclass reformulation in all three datasets. However, as stated before, this decline should be interpreted cautiously because the multiclass setting is a harder prediction problem, uses different averaging over classes, and relies on heuristic labels derived from dominant proxy activations. Therefore, the reduction in macro F1 does not by itself establish hidden failure modes or reveal ground-truth attack semantics. Instead, it indicates that the baseline separates the proposed behavioral proxies unevenly once the task requires distinguishing among them. The effect is most pronounced in SWaT and WADI. In SWaT, macro F1 decreases from 85.44% under binary classification to 37.84% under multiclass classification. In WADI, it decreases from 85.21% to 42.92%. HAI shows a smaller
Title Suppressed Due to Excessive Length
15
reduction, from 76.67% to 58.01%, suggesting comparatively stronger separability among the proxy behaviors. Overall, the binary setting shows that the baseline can reasonably well separate attacks from normal windows. In contrast, the multiclass reformulation shows that the same representation does not uniformly separate the behavioral proxies. This makes the multiclass experiment a useful descriptive complement to binary evaluation, but only as a proxy-based separability analysis rather than evidence that binary evaluation hides detection failures.
5
Key Takeaways
This study introduces a framework for characterizing ICS attack traces using five interpretable temporal primitives: drift, spike, oscillation, repetition, and switching. Applied to SWaT, WADI, and HAI, it shows that attack windows are not a single homogeneous class, but instead exhibit diverse temporal patterns across and within datasets. The framework is intended as a behavioral description, not a taxonomy of attack semantics. A second contribution is the comparison of dataset coverage in this normalized behavioral space. The results suggest that common ICS benchmarks emphasize different temporal morphologies, so evaluation on one dataset may overrepresent some behavioral profiles and underrepresent others. These differences should be interpreted as normalized behavioral shape, not as direct comparisons of physical severity or operational impact. A third contribution is proxy-based behavior stratification as a complement to binary IDS evaluation. The multiclass reformulation is not evidence that binary detection hides failures; rather, it shows how performance varies when the task requires separating heuristic behavioral proxies instead of only attack versus normal. In this sense, the framework adds a lens for studying dataset bias and coverage, rather than replacing standard benchmarking.
6
Limitations
Several limitations should be noted. First, the framework describes observable temporal deviations, but it does not infer attacker intent, semantic attack categories, or causal mechanisms. Second, it uses only L0/L1 process data, so attacks that remain purely cyber and do not affect physical variables cannot be captured. Third, the five primitives were chosen for interpretability and portability, not exhaustiveness. Other descriptors, including spectral, entropy-based, or control-aware features, may capture behaviors outside the current vocabulary. The fixed 60-second window may also smooth short transients and truncate slow degradations. Fourth, the single dominant-label assignment is a coarse heuristic that can hide mixed behaviors when multiple primitives co-occur in the same window. Finally, the primitives are computed on normalized signals and aggregated across variables, so they describe unitless temporal morphology rather than directly comparable physical severity. In other words, similar primitive values may correspond to different operational phenomena across datasets. Hence,
16
V. Kampourakis et al.
cross-dataset comparisons should be interpreted as comparisons of normalized behavioral shape, not physical impact.
7
Conclusion
ICS security relies heavily on benchmark datasets to train and evaluate IDS. However, evaluating these benchmarks only through homogeneous binary labels provides an incomplete view of IDS performance, especially when attack traces differ substantially in their temporal manifestations. In this paper, we propose a behavioral characterization framework to examine normalized temporal morphology across SWaT, WADI, and HAI. Our analysis showed that cyber-physical attack windows exhibit diverse process-trace patterns, including abrupt spikes, sustained drifts, sensor freezing, and forced actuator switching, and that standard benchmarks differ in the normalized behavioral profiles they emphasize. These findings indicate that aggregate binary metrics may obscure how performance varies across different proxy-based attack manifestations. At the same time, we do not argue against binary detection itself; rather, we view it as a well-established first stage in a broader escalation pipeline. Our critique is directed at the evaluation practice that treats binary performance as sufficient evidence of robustness, even though behavior-aware triage and diagnosis are rarely instantiated or assessed. Accordingly, our framework is meant to complement binary evaluation with behavior-stratified benchmarking, so that variation in normalized temporal morphology is measured and not assumed away. Promising avenues for future work include: i) to expand the behavioral vocabulary with richer descriptors, such as spectral, entropy-based, delay-sensitive, or control-aware features; ii) to move from single-label assignment toward mixed-label or multi-label representations to better reflect co-occurring behaviors and reduce winner-take-all effects; iii) to broader evaluate across more datasets, and ideally, expert-curated annotations to facilitate test generalization and strengthen validation; and iv) to integrate behavioral characterization with IDS triage, alert prioritization, and incident response to make the framework more operationally useful.
A
Sensitivity Analysis
To assess the sensitivity of the proposed framework to parameter choices, we vary two key parameters: the sliding-window length and the dominant-property threshold. Window lengths of 30, 60, and 120 samples are combined with threshold values τ ∈ {0.2, 0.3, 0.4}. For each configuration, we report three metrics. First, we quantify the stability of the dominant behavioral labeling across parameter settings by measuring the agreement rate with the selected operating point (w = 60, τ = 0.30). Adom =
M 1 X I p̂j = p̂base , j M j=1
(10)
Title Suppressed Due to Excessive Length
17
where p̂j denotes the dominant behavioral property assigned to the j-th is the corresponding dominant attack window under a given configuration. p̂base j property under the baseline configuration (w = 60, τ = 0.30), M is the number of matched windows, and I[·] is the indicator function, which equals 1 when the two labels match and 0 otherwise. Higher values of Adom indicate that the same attack window is assigned the same dominant behavioral label across parameter changes, whereas lower values indicate greater sensitivity to the chosen window size or threshold. Second, behavioral diversity is measured via entropy, as shown in equation 11. H=−
N X
pi log(pi ),
(11)
i=1
where pi is the proportion of windows whose dominant property is i and N is the number of behavioral categories. Specifically, H summarizes how many primitives are actually exercised, namely, low entropy means that attacks are dominated by one or two behaviors (e.g., mostly spikes), whereas high entropy indicates a richer mix of behaviors (spikes, drifts, oscillations, etc.). Second, we quantify the stability of behavioral diversity relative to the selected operating point (w = 60, τ = 0.30). SH = 1 −
|H − Hbase | , log(N )
(12)
where Hbase is the entropy at (w = 60, τ = 0.30). Values SH ≈ 1 indicate that the behavioral diversity remains close to the baseline under parameter changes, while lower values indicate higher sensitivity. As observed in Table 5, entropy remains relatively stable for SWaT and HAI, with only modest changes across window lengths and threshold values. WADI exhibits a stronger dependence on window length, with entropy decreasing for larger windows and increasing for shorter windows, indicating that temporal granularity influences the diversity of dominant behavioral assignments. In parallel, the agreement rate Adom is highest at the selected configuration (w = 60, τ = 0.30) and remains very close to 1.0 for nearby settings in SWaT and HAI, confirming that the dominant behavioral label is highly stable in these configurations. WADI also achieves perfect agreement at the baseline configuration and maintains relatively high agreement under larger windows, although agreement decreases more noticeably for shorter windows. Overall, the selected configuration represents a balanced operating point, providing highly stable labeling across all three datasets while minimizing sensitivity to parameter changes. Acknowledgments. This work is supported by the Research Council of Norway through the SFI Norwegian Centre for Cybersecurity in Critical Sectors (NORCICS) project no. 310105 and by the European Union’s Horizon Europe Research and Innovation Programme through SECASSURED under grant agreement No 101225858 and ENFIELD under grant agreement No 101120657.
18
V. Kampourakis et al.
Table 5: Sensitivity analysis for window size and dominant-property threshold. Highlighted rows indicate the parameter configuration used in the experiments of Section 4. Adom Entropy SH
30 30 30 60 60 60 120 120 120
0.808 0.807 0.800 1.000 1.000 0.998 0.893 0.893 0.893
0.798 0.796 0.790 1.000 1.000 0.991 0.865 0.865 0.859
0.631 0.630 0.625 1.000 1.000 0.999 0.808 0.808 0.808
0.2 0.3 0.4 0.2 0.3 0.4 0.2 0.3 0.4
0.509 0.521 0.555 0.336 0.336 0.346 0.308 0.308 0.314
0.903 0.897 0.878 1.000 1.000 0.994 0.984 0.984 0.988
0.498 0.512 0.520 0.328 0.329 0.329 0.301 0.301 0.302
0.906 0.898 0.893 0.999 1.000 1.000 0.984 0.984 0.985
WADI
Adom Entropy SH
HAI
Adom Entropy SH
SWaT
Window τ
0.658 0.660 0.665 0.802 0.802 0.801 0.468 0.468 0.468
0.696 0.695 0.689 0.853 0.853 0.852 0.797 0.797 0.797
References 1. Adjewa, F., Esseghir, M., Merghem-Boulahia, L., Kacfah, C.: Llm-based continuous intrusion detection framework for next-gen networks. In: 2025 International Wireless Communications and Mobile Computing (IWCMC). pp. 1198–1203 (2025). https://doi.org/10.1109/IWCMC65282.2025.11059643 2. Ahmed, C.M., Palleti, V.R., Mathur, A.P.: Wadi: a water distribution testbed for research in the design of secure cyber physical systems. In: Proceedings of the 3rd international workshop on cyber-physical systems for smart water networks. pp. 25–28 (2017). https://doi.org/https://doi.org/10.1145/3055366.3055375 3. Ahmed, W.: Hybrid ensemble method for detecting cyber-attacks in water distribution systems using the batadal dataset. arXiv preprint arXiv:2512.14422 (2025) 4. Clark-Carter, D.: Interquartile Range. John Wiley & Sons, Ltd (2005). https: //doi.org/https://doi.org/10.1002/0470013192.bsa311 5. Conti, M., Donadel, D., Turrin, F.: A survey on industrial control system testbeds and datasets for security research. IEEE Communications Surveys & Tutorials 23(4), 2248–2294 (2021). https://doi.org/10.1109/COMST.2021.3094360 6. Hu, Y., Yang, A., Li, H., Sun, Y., Sun, L.: A survey of intrusion detection on industrial control systems. International Journal of Distributed Sensor Networks 14(8), 1550147718794615 (2018) 7. Jaradat, S., Komol, M.M., Elhenawy, M., Dong, N.: Cyberattack detection on swat plant industrial control systems using machine learning. Artificial Intelligence and Autonomous Systems 1(2) (2024). https://doi.org/10.55092/aias20240006 8. Kadosh, N., Frid, A., Housh, M.: Detecting cyber-physical attacks in water distribution systems: One-class classifier approach. Journal of Water Resources Planning and Management 146(8), 04020060 (2020). https://doi.org/10.1061/(ASCE)WR. 1943-5452.0001259 9. Kampourakis, K.E., Gkioulos, V., Katsikas, S.: Cybersecurity digital twins for industrial systems: From literature synthesis to framework design. Information 17(3) (2026). https://doi.org/10.3390/info17030286 10. Kampourakis, K.E., Gkioulos, V., Katsikas, S.: Systematic integration of digital twins and constrained llms for interpretable cyber-physical anomaly detection. In: ICT Systems Security and Privacy Protection. pp. 199–212. Springer
Title Suppressed Due to Excessive Length
19
Nature Switzerland, Cham (2026). https://doi.org/https://doi.org/10.1007/ 978-3-032-27993-4_14 11. Krotofil, M., Cárdenas, A.A., Manning, B., Larsen, J.: Cps: driving cyber-physical systems to unsafe operating conditions by timing dos attacks on sensor signals. In: Proceedings of the 30th Annual Computer Security Applications Conference. p. 146–155. ACSAC ’14, Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2664243.2664290 12. Lamberts, O., Wolsing, K., Wagner, E., Pennekamp, J., Bauer, J., Wehrle, K., Henze, M.: Sok: Evaluations in industrial intrusion detection research. arXiv preprint arXiv:2311.02929 (2023) 13. Lemay, A., Fernandez, J.M.: Providing SCADA network data sets for intrusion detection research. In: 9th Workshop on Cyber Security Experimentation and Test (CSET 16). USENIX Association, Austin, TX (Aug 2016), https://www.usenix. org/conference/cset16/workshop-program/presentation/lemay 14. Lian, W., Zhang, C., Zhang, H., Jia, B., Liu, B.: Rulemaster+: Llm-based automated rule generation framework for intrusion detection systems. Chinese Journal of Electronics 34(5), 1402–1415 (2025). https://doi.org/10.23919/cje.2024. 00.342 15. MacFarland, T.W., Yates, J.M.: Mann–Whitney U Test, pp. 103–132. Springer International Publishing, Cham (2016). https://doi.org/10.1007/ 978-3-319-30634-6_4 16. Mathur, A.P., Tippenhauer, N.O.: Swat: a water treatment testbed for research and training on ics security. In: 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater). pp. 31–36 (2016). https://doi. org/10.1109/CySWater.2016.7469060 17. Mustafa, A., Khan, M.T., Umer, M.A., Masood, Z., Ahmed, C.M.: Adversarial sample generation for anomaly detection in industrial control systems. In: Proceedings of the 1st Workshop on Modeling and Verification for Secure and Performant Cyber-Physical Systems. MoVe4SPS ’25, Association for Computing Machinery, New York, NY, USA (2025). https://doi.org/10.1145/3735948.3736158 18. Oyama, H., Rangan, K.K., Durand, H.: Handling of stealthy sensor and actuator cyberattacks on evolving nonlinear process systems. Journal of Advanced Manufacturing and Processing 3(3), e10099 (2021). https://doi.org/https://doi.org/ 10.1002/amp2.10099 19. Radoglou-Grammatikis, P., Sarigiannidis, P., Efstathopoulos, G., Karypidis, P.A., Sarigiannidis, A.: Diderot: an intrusion detection and prevention system for dnp3based scada systems. In: Proceedings of the 15th International Conference on Availability, Reliability and Security. ARES ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3407023.3409314 20. Schmidl, S., Wenig, P., Papenbrock, T.: Anomaly detection in time series (2022). https://doi.org/10.25932/publishup-66366 21. Shen, Y., Qin, Z.: Detection, differentiation and localization of replay attack and false data injection attack based on random matrix. Scientific Reports 14(1), 2758 (2024). https://doi.org/10.1038/s41598-024-52954-z 22. Shin, H.K., Lee, W., Yun, J.H., Kim, H.: HAI 1.0: HIL-based augmented ICS security dataset. In: 13th USENIX Workshop on Cyber Security Experimentation and Test (CSET 20). USENIX Association (Aug 2020), https://www.usenix.org/ conference/cset20/presentation/shin 23. Taormina, R., et. al.: Battle of the attack detection algorithms: Disclosing cyber attacks on water distribution networks. Journal of Water Resources Planning
20
V. Kampourakis et al.
and Management 144(8), 04018048 (2018). https://doi.org/10.1061/(ASCE)WR. 1943-5452.0000969 24. Umer, M.A., Junejo, K.N., Jilani, M.T., Mathur, A.P.: Machine learning for intrusion detection in industrial control systems: Applications, challenges, and recommendations. International Journal of Critical Infrastructure Protection 38, 100516 (2022). https://doi.org/https://doi.org/10.1016/j.ijcip.2022.100516 25. Urbina, D.I., Urbina, D.I., Giraldo, J., Cardenas, A.A., Valente, J., Faisal, M., Tippenhauer, N.O., Ruths, J., Candell, R., Sandberg, H.: Survey and new directions for physics-based attack detection in control systems. US Department of Commerce, National Institute of Standards and Technology . . . (2016) 26. Zamanzadeh Darban, Z., Webb, G.I., Pan, S., Aggarwal, C., Salehi, M.: Deep learning for time series anomaly detection: A survey. ACM Comput. Surv. 57(1) (Oct 2024). https://doi.org/10.1145/3691338 27. Zambrano, A., Betancur, A.P., Burbano, L., Niño, A.F., Giraldo, L.F., Soto, M.G., Giraldo, J., Cardenas, A.A.: You make me tremble: A first look at attacks against structural control systems. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. p. 1320–1337. CCS ’21, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10. 1145/3460120.3485386