arXiv:2605.13337v1 [cs.CR] 13 May 2026
Context-Aware Web Attack Detection in Open-Source SIEM Systems via MITRE ATT&CK-Enriched Behavioral Profiling Badr Alboushy1*, Assef Jafar2 , Mohamad Aljnidi3 , Mohamad Bashar Disoki1 , Aref Shaheed4 1*
Higher Institute for Applied Sciences and Technology (HIAST), Damascus, Syria. 2 Syrian Private University, Damascus, Syria. 3 Arab International University, Damascus, Syria. 4 Latakia University, Latakia, Syria.
*Corresponding author(s). E-mail(s): [email protected]; Contributing authors: [email protected]; [email protected]; [email protected]; [email protected]; Abstract Security Information and Event Management (SIEM) systems aggregate log data from heterogeneous network devices and applications in order to detect coordinated attacks that stateless per-event analysis cannot reveal — multi-step attack campaigns unfold across sequences of events from a single source, requiring intra-source temporal correlation rather than inter-source aggregation alone. Traditional rule-based correlation engines embedded in open-source SIEM platforms are effective at detecting individually recognisable anomalies yet struggle to classify multi-step, low-signal web application attacks because they examine each event without reference to the behavioural history of the originating host. This paper presents Smart-SIEM, an AI module designed as a modular enhancement for the open-source Wazuh SIEM platform. The module requires configuring Wazuh rule levels to route events to the classification pipeline; in regulated environments this configuration step may require compliance review. The module introduces two contributions. First, we define a per-source-IP behavioural context vector that summarises the most recent N security events from the same host, encoding the distribution of HTTP response-status buckets, the maximum rule activation count, and the accumulation of MITRE ATT&CK technique identifiers observed in prior events. Second, we deploy a two-stage cascade in which
1
Stage 1 distinguishes normal from malicious traffic and Stage 2 assigns one of six fine-grained attack labels (SQL Injection, XSS, Web Vulnerability Scanning, Brute Force, Broken Authentication, Sensitive Data Exposure). We construct a purpose-built labelled dataset of 46,454 Wazuh security events collected from a controlled testbed in which the OWASP Juice Shop application serves as the victim, with simultaneous benign Selenium-driven traffic and adversarial traffic generated by SQLMAP, Acunetix, Burp Suite, and an XSS automation tool, each originating from distinct IP addresses. Ground-truth labels are assigned deterministically from IP identity and recorded attack timestamps. A comparative evaluation of eight gradient boosting and baseline algorithms on a single-session-per-class testbed (reported F1 values therefore represent an upper bound on generalisation to unseen attack campaigns) demonstrates that without context features all algorithms converge to ≈0.705 macro F1 ; with context features they rise to 0.947–0.967 (Stage 1) and 0.876–0.914 (Stage 2) — an average improvement of +0.254 and +0.324 respectively. This algorithm-agnostic improvement confirms that the behavioural context vector is the primary contribution. A hybrid cascade combining LightGBM for Stage 1 and XGBoost for Stage 2 achieves the best overall performance: F1 of 0.967 (binary) and 0.914 (six-class). An ablation study over N ∈ {3, . . . , 35} identifies N = 30 as a practical operating point. Wazuh’s native rule engine detects 0% of Brute Force and Broken Authentication events; the AI module detects 100% and 98.3% respectively. A self-adaptive retraining mechanism demonstrates recovery from concept drift: F1 drops from 0.905 to 0.465 when unseen attack types emerge, triggering retraining; partial recovery to 0.695 (+0.099) with Phase 2-only retraining; the production-intended Phase 1+2 protocol recovers to 0.814 at the cost of a modest regression on originally-known classes. Keywords: Security Information and Event Management (SIEM), Intrusion Detection, Gradient Boosting, Hybrid Cascade Classification, Event Correlation, MITRE ATT&CK, Behavioural Profiling, Web Application Security, Contextual Feature Engineering, Self-Adaptive Systems
1 Introduction The proliferation of web-facing services has made web application attacks among the most prevalent and damaging threats facing organisations today González-Granadillo et al. (2021). SQL injection, cross-site scripting (XSS), credential stuffing, and automated vulnerability scanning each manifest as extended campaigns rather than isolated events: an attacker probes hundreds of endpoints, enumerates directories, and escalates privilege across multiple sessions before a successful breach is complete. Detecting such multi-step campaigns requires correlating security events across time and across the behavioural fingerprint of a given source address—a task that exposes the fundamental limitation of purely rule-based SIEM correlation engines. Security Information and Event Management (SIEM) platforms occupy a central role in enterprise security operations. They aggregate log data from firewalls, intrusion detection systems (IDS), web servers, and endpoints, normalise heterogeneous formats,
2
apply correlation rules, and raise alerts when rule conditions are satisfied. Open-source platforms such as Wazuh Wazuh, Inc. (2022) provide a mature rule engine aligned with standards including PCI DSS, HIPAA, NIST 800-53, GDPR, and the MITRE ATT&CK framework Strom et al. (2018). Yet the rule-engine paradigm has three wellknown weaknesses when confronted with sophisticated web attacks: (1) rules depend on expert-curated signatures that cannot anticipate zero-day or polymorphic payloads; (2) each event is evaluated independently, so gradual reconnaissance patterns that unfold over dozens of low-severity events are missed; and (3) the false positive burden is high because many benign activities trigger the same low-level rule groups used as indicators of attack. Machine learning offers a complementary paradigm. Rather than matching against fixed signatures, a trained classifier can learn statistical boundaries between normal and malicious event sequences. However, prior ML-based intrusion detection research largely targets network-packet datasets (KDD 99, NSL-KDD, CICIDS) rather than SIEM log events, and almost universally treats each record as an independent observation rather than as part of an ongoing session or campaign Tavallaee et al. (2009); Sharafaldin et al. (2018). This paper bridges that gap with three concrete contributions: 1. A contextual behavioural feature set for SIEM events. For each incoming security event we construct a feature vector that aggregates the preceding N events from the same source IP, encoding HTTP response-code distributions, peak rule activation frequency, and the cumulative count of each MITRE ATT&CK technique identifier observed in the history window. This transforms a stateless event classifier into a session-aware detector. 2. A two-stage hybrid cascade classifier. Stage 1 (LightGBM) makes a binary NORMAL/ATTACK decision. Only events flagged as ATTACK proceed to Stage 2 (XGBoost), which resolves the fine-grained attack category. A comparative study of eight algorithms demonstrates that the context vector improves all tested algorithms by +0.25–+0.35 F1 , establishing the context features rather than any specific algorithm as the primary contribution. 3. A self-adaptive retraining mechanism. The system maintains a labelled knowledge base in Elasticsearch. Security analysts can add or relabel events; the system automatically evaluates current model accuracy against the knowledge base and retrains when accuracy falls below 90%, allowing the deployed classifier to evolve with the monitored environment. The remainder of this paper is organised as follows. Section 2 reviews related work. Section 3 provides background on SIEM architecture, gradient boosting classifiers, and SMOTE-NC. Section 4 describes the Smart-SIEM architecture. Section 5 details the dataset construction methodology. Section 6 defines the contextual feature engineering procedure. Section 7 presents the cascaded classifier design and training protocol. Section 8 reports experimental results and ablation studies. Section 9 discusses implications and limitations. Section 10 concludes.
3
2 Related Work 2.1 Rule-Based SIEM Correlation Security Information and Event Management (SIEM) systems arose from the convergence of Security Information Management (SIM) and Security Event Management (SEM), and their operational role in enterprise security operations has been well established for over a decade Miller et al. (2010); Bhatt et al. (2014). The foundational purpose of a SIEM is to correlate events from disparate sources in order to infer higher-level security states that no single event reveals in isolation Müller et al. (2009). Chuvakin et al. (2012) provide a comprehensive treatment of log management, emphasising that the quality and completeness of collected log data is a prerequisite for any correlation-based detection strategy. Rule-based correlation engines encode this knowledge as condition-action pairs: when a set of logical predicates over event fields is satisfied within a configurable time window, an alert is raised Miller et al. (2010). Agrawal and Makwana Agrawal and Makwana (2015) survey the critical capabilities of SIEM platforms and conclude that rule quality and maintenance burden are the primary bottlenecks to accurate detection in practice. González-Granadillo et al. (2021) compare eight commercial and open-source SIEM products (ArcSight, QRadar, McAfee SIEM, LogRhythm, USM-OSSIM, RSA NetWitness, Splunk, SolarWinds) across seventeen capability dimensions and find that open-source alternatives lag commercial offerings primarily in data analytics, User and Entity Behaviour Analytics (UEBA), and risk-analysis depth—capabilities that machine learning can address without commercial licensing costs. Despite their widespread adoption, rule-based engines face three structural limitations: they depend on expert-curated signatures that cannot anticipate zero-day or polymorphic payloads Stallings and Brown (2017); each event is evaluated independently, so gradual reconnaissance campaigns that unfold over dozens of low-severity events remain invisible; and the false positive burden is high because broad rule groups match many benign activities Garcia-Teodoro et al. (2009).
2.2 ML-Based Intrusion Detection The application of machine learning to intrusion detection has a long history, tracing back to the seminal statistical anomaly model of Denning (1987), which established the conceptual foundation for behaviour-based detection. Liao et al. (2013) and Buczak and Guven (2016) provide extensive surveys of the subsequent literature, cataloguing techniques ranging from naı̈ve Bayes and support vector machines to neural networks and ensemble methods. The benchmark datasets that underpin much of this literature—including KDD 99, its refined successor NSL-KDD Tavallaee et al. (2009), and the more recent CICIDS-2017 Sharafaldin et al. (2018)—represent network-flow features rather than SIEM-normalised security events, which limits their direct applicability inside a SIEM pipeline. Sommer and Paxson (2010) articulate a fundamental tension: the high-dimensional, imbalanced, and concept-drifting nature of real network traffic makes the standard evaluation assumptions of machine learning research— closed-world classifiers, stationary distributions, balanced classes—frequently invalid 4
in operational security settings. Ensemble methods, in particular gradient boosting variants Chen and Guestrin (2016); Ke et al. (2017); Breiman (2001), consistently dominate classification benchmarks owing to their robustness to irrelevant features and their ability to model non-linear interaction effects without manual feature engineering. Deep learning approaches—LSTM networks, convolutional models, and attention-based architectures—have also shown strong performance on sequential network data Ferrag et al. (2020); Vinayakumar et al. (2019), although they typically require larger labelled datasets and offer less interpretability than gradient boosting on tabular inputs. Chandola et al. (2009) provide a unifying taxonomy of anomaly detection methods and highlight that the scarcity of labelled anomalies, combined with severe class imbalance, remains a persistent obstacle to supervised detection—a challenge directly addressed in this work through SMOTE-NC oversampling Chawla et al. (2002); Fernández et al. (2018); He and Garcia (2009); Guo et al. (2017).
2.3 Web Application Attack Detection Web application attacks represent a distinct sub-domain of intrusion detection because the relevant signals appear at the HTTP application layer rather than in network packet headers. SQL injection—the systematic injection of database commands through user-supplied input fields—and cross-site scripting (XSS)—the injection of executable client-side code—remain among the most prevalent and damaging attack vectors Halfond et al. (2006); Grossman (2007); OWASP Foundation (2021a). The OWASP Web Security Testing Guide OWASP Foundation (2021b) catalogs a broader taxonomy of web vulnerabilities and provides a reference framework widely adopted in both academic and industrial testing. Detecting these attacks from server-side logs is challenging because individual malicious requests are often syntactically indistinguishable from benign ones; meaningful signal emerges only when considering the sequence of requests from a given source address Nisioti et al. (2018); Chandola et al. (2009). The MITRE ATT&CK framework Strom et al. (2018) provides a structured vocabulary of adversary techniques, and recent work on cyber threat intelligence extraction Husari et al. (2017) and threat hunting Milajerdi et al. (2019) has demonstrated that ATT&CK technique identifiers can serve as high-level behavioural signatures that discriminate attack campaigns across diverse data sources. To the best of our knowledge, no prior work has accumulated ATT&CK technique frequencies across a per-IP event history window within a SIEM pipeline and used those cumulative counts as discriminative classifier features.
2.4 ML-Based SIEM Enhancement Several works have attempted to augment SIEM platforms with machine learning. Veeramachaneni et al. (2016) propose an AI2 framework combining unsupervised anomaly detection with analyst feedback loops; their system operates on generic log streams rather than SIEM-normalised event formats and requires sustained analyst labelling effort to function effectively. Sarker et al. (2020) apply decision tree ensembles to log-derived features but treat each event as an independent observation, discarding the temporal context that distinguishes reconnaissance campaigns from 5
isolated probes. Ring et al. (2019) survey network-based intrusion detection datasets and highlight the near-complete absence of SIEM-native labelled collections, while Kwon et al. (2019) review deep learning for log-based anomaly detection but focus on system logs rather than security-event correlation. Creech and Hu (2014) demonstrate that semantic features derived from system call sequences substantially improve detection of host-level intrusions, providing an analogy for the hypothesis explored here: that behavioural sequences of SIEM events carry information that point-intime features do not. Interpretability of learned models is increasingly recognised as a deployment requirement in security operations Bhatt et al. (2014). We address this through gain-based feature importance analysis (Section 8), which confirms that context features dominate Stage 1 discrimination. Concept drift Gama et al. (2014); Lu et al. (2019); Yang et al. (2021); Pendlebury et al. (2019) motivates the self-adaptive retraining mechanism described in Section 4. Table 1 situates Smart-SIEM against the most closely related prior systems on six dimensions relevant to operational SIEM deployment.
Table 1: Comparison of representative ML-based intrusion detection and SIEMaugmentation systems. ✓ = explicitly present; ◦ = partial or indirect; × = absent. Context features : whether the system aggregates temporal behavioural signals from prior events. ATT&CK : whether MITRE ATT&CK technique identifiers are used as input features. Drift : whether the system includes a mechanism to adapt to concept drift without full retraining. SIEM platform : whether the system integrates natively with a deployed SIEM product. System
Year
Data Format
Context Features
ATT&CK
AI2 Veeramachaneni et al. (2016) IntruDTree Sarker et al. (2020) DeepLog Ferrag et al. (2020) Nisioti et al. (2018) CADE Yang et al. (2021)
2016 2020 2020 2018 2021
Generic logs Network flows System logs IDS alert streams Network flows
◦ time-window agg. × per-event only ◦ LSTM (implicit) ◦ multi-stage corr. × per-event only
× × × × ×
Smart-SIEM (ours)
2022
Wazuh SIEM events
✓ per-IP N =30 hist.
✓ 7 tech.
Drift
SIEM
Task
◦ analyst feedback × none ◦ incremental × none ✓ autoencoder
× × × ◦ generic ×
Binary anomaly Multi-class IDS Binary anomaly Attack attribution Anomaly + drift
✓ threshold
✓ Wazuh
2-stage cascade
Three differentiators are evident. First, no prior system operates on Wazuhnormalised security events; all existing work uses network flow datasets (NSL-KDD, CICIDS) or generic log streams that do not preserve compliance mappings, MITRE ATT&CK identifiers, or the rule-correlation metadata native to SIEM platforms. Second, the context vector in Smart-SIEM is the only proposal to use cumulative MITRE ATT&CK technique counts as explicit contextual features — existing context-aware approaches (AI2 , DeepLog) aggregate raw signal statistics without the threat-framework semantics. Third, the self-adaptive retraining loop in Smart-SIEM is the only drift-adaptation mechanism that operates at the knowledge-base accuracy level, allowing non-expert operators to trigger retraining by labelling a small set of analyst-reviewed events, without requiring a separate drift-detection model Yang et al. (2021); Pendlebury et al. (2019). 6
3 Background 3.1 SIEM Architecture A SIEM platform comprises six functional components Miller et al. (2010); Bhatt et al. (2014); Chuvakin et al. (2012): (1) data sources —network devices, servers, and applications that generate log records; (2) log collection —agents or syslog receivers that transport events to a centralised processor; (3) parsing and normalisation —extraction of structured fields from free-form log text and mapping to a canonical schema; (4) correlation engine —rule-based or analytics-driven logic that identifies patterns across multiple events; (5) log storage —indexed persistence for forensic analysis and compliance reporting; and (6) monitoring and visualisation —dashboards and alerting interfaces for security analysts. In Wazuh, the correlation engine applies XML-encoded rules to decoded event fields. Each rule carries a severity level (0–16), a frequency counter that tracks how many times it has fired within a rolling time window, compliance mappings (PCI DSS, HIPAA, NIST, GDPR), and a MITRE ATT&CK technique identifier. Events that match a rule with level > 0 are forwarded to Elasticsearch via Filebeat, where they are stored as JSON documents in the wazuh-alerts-* index pattern.
3.2 Gradient Boosting Classifiers The proposed hybrid uses LightGBM Ke et al. (2017) for Stage 1 and XGBoost Chen and Guestrin (2016) for Stage 2; CatBoost Prokhorenkova et al. (2018); Dorogush et al. (2018) is included as a comparison baseline. LightGBM uses a histogram-based leaf-wise growth strategy that achieves fast training on large datasets; it is the Stage 1 choice because it achieves the highest binary F1 (0.967) in our comparison. XGBoost uses a level-wise growth strategy with a regularised objective; it is the Stage 2 choice because it achieves the highest multi-class F1 (0.914). CatBoost is a gradient-boosting algorithm developed by Yandex that introduces two key innovations over earlier approaches such as XGBoost Chen and Guestrin (2016) and LightGBM. First, it builds symmetric (oblivious) decision trees where every node at the same depth applies the same feature split, which reduces prediction latency by transforming tree traversal into a single vectorised comparison. Second, it employs ordered target encoding to eliminate prediction shift—a systematic bias that arises in standard target encoding when the statistic for a category is computed from examples that include the current one. LightGBM Ke et al. (2017) uses gradient-based oneside sampling (GOSS) and exclusive feature bundling (EFB) to reduce training time while maintaining accuracy. XGBoost Chen and Guestrin (2016) uses second-order gradient statistics and regularised tree learning with column sub-sampling. Among the evaluated algorithms, CatBoost accepts categorical features in their raw string form without the need for one-hot encoding, which makes it well-suited for SIEM data where many fields (rule groups, compliance tags, MITRE identifiers) are naturally categorical with high cardinality. 7
3.3 SMOTE-NC Class imbalance is endemic in security datasets Guo et al. (2017); He and Garcia (2009) and SIEM deployments are no exception: normal traffic vastly outnumbers labelled attack events in real deployments. The Synthetic Minority Over-sampling Technique (SMOTE) Chawla et al. (2002) addresses this by generating synthetic minority-class instances by interpolating between each minority sample and one of its k -nearest neighbours in feature space. SMOTE-NC (Nominal and Continuous) extends SMOTE to mixed categorical-numerical datasets Chawla et al. (2002): for categorical features, the new instance inherits the most frequent value among the k neighbours, while the standard Euclidean distance is augmented by the median standard deviation of all numerical columns to account for categorical mismatches. This prevents the nonsensical interpolation that naive numeric encoding of categories would produce.
4 System Architecture Smart-SIEM is architected as a loosely coupled module that attaches to an existing Wazuh deployment without modifying its core components. Figure 1 shows the overall data flow.
Smart-SIEM AI Module
Producer
pull events Apache Kafka
Consumer Workers
classify
AI Hybrid Cascade Stage 1: LightGBM (Binary) Stage 2: XGBoost (Multi-class)
Syslog Wazuh Agent (Web Server)
Wazuh Agent (App Server)
Wazuh Agent · · ·
context query
Wazuh Manager Cluster Filebeat
Elasticsearch
wazuh-alerts-*
wazuh-aihybrid-*
Kibana Dashboard
Fig. 1: Architecture of Smart-SIEM. Security events produced by Wazuh agents are processed by the Wazuh Manager cluster, which forwards events simultaneously to Elasticsearch (standard wazuh-alerts-* index via Filebeat) and to the AI module via syslog. The AI module enqueues events in Apache Kafka; consumer workers retrieve per-IP context from Elasticsearch and classify each event with the hybrid cascade model. Classification results are written to a dedicated wazuh-ai-hybrid-* index and visualised in a custom Kibana dashboard.
8
4.1 Wazuh Manager Cluster The Wazuh Manager cluster is configured to emit every security event whose rule level exceeds zero as a syslog message directed to the AI module’s producer listener. To ensure that low-severity (level = 0) events—which Wazuh normally silences—also reach the AI module, we applied a bulk rule transformation that promotes all level = 0 rules to level = 1. These benign events carry valuable negative-class signal and their rule-field metadata (groups, MITRE identifiers, compliance tags) helps the classifier characterise normal behaviour.
4.2 Apache Kafka Message Queue Each Wazuh Manager node runs a Python producer process that parses the incoming syslog stream, deserialises the JSON payload, and publishes the event to a Kafka topic. Kafka provides back-pressure handling, at-least-once delivery guarantees, and horizontal fan-out to multiple consumer groups, satisfying the scalability and faulttolerance requirements of a production SIEM pipeline.
4.3 Consumer and Classification Pipeline Consumer workers pull messages from Kafka and execute the following pipeline for each event: 1. Preprocessing. Missing categorical fields are filled with empty strings; missing numerical fields receive default values (rule level: 3, firedtimes: 1). The rule.mail boolean is mapped to {0, 1}. 2. Context retrieval. The consumer queries Elasticsearch for the N most recent events whose data.srcip matches the current event’s source IP and whose timestamp precedes the current event. The context window N is a configurable parameter; our ablation study (Section 8.4) identifies N = 30 as the optimal value. 3. Context feature construction. From the retrieved history the consumer computes the context feature vector (defined formally in Section 6) and appends it to the current event’s feature vector. 4. Stage 1 classification. The enriched feature vector is passed to the LightGBM binary classifier (Stage 1). If the prediction is NORMAL, the event is written directly to the output index. 5. Stage 2 classification. Events predicted as ATTACK by Stage 1 are passed to the multi-class XGBoost classifier (Model 2), which assigns one of six attack labels. 6. Result persistence. The classified event, including both Stage 1 and Stage 2 labels, is written to the wazuh-ai-hybrid-* Elasticsearch index.
4.4 Self-Adaptive Retraining To maintain accuracy as the monitored environment evolves, the system maintains a labelled knowledge base index (wazuh-ai-knowledge base) in Elasticsearch. Analysts may add new labelled events or correct existing labels via a dedicated interface. Each 9
day, the consumer checks classifier accuracy against the full knowledge base. If macroaveraged accuracy falls below 90%, new models are trained on the updated knowledge base and deployed, replacing the previous models. The 90% threshold is configurable. Hybrid Cascade Models (LightGBM Stage 1 + XGBoost Stage 2)
Classify Incoming Security Events
Kibana Dashboard (Analyst Review Interface)
Knowledge Base (wazuh-ai-knowledge base) Analyst adds / corrects labels
Daily Accuracy Evaluation (macro-averaged, vs. full KB)
Threshold: 90% (configurable)
YES
Accuracy ≥ 90%?
Continue — no model update
NO Trigger Retraining on Updated KB
Deploy New Models (replace production)
Fig. 2: Self-adaptive retraining loop. The hybrid cascade models classify events continuously; results are surfaced in a Kibana dashboard where security analysts can add or correct labels, which are stored in the wazuh-ai-knowledge base Elasticsearch index. Each day the system evaluates macro-averaged classifier accuracy against the full knowledge base; if accuracy falls below the 90% threshold, both Stage 1 and Stage 2 models are retrained on the updated knowledge base and the new models replace the current production versions.
5 Dataset Construction 5.1 Testbed Configuration Because no publicly available labelled SIEM dataset exists that captures multi-class web application attacks in Wazuh’s event format, we constructed a purpose-built dataset using a controlled testbed (Figure 3). The testbed consists of:
• Victim server. OWASP Juice Shop Kimminich (2021), a deliberately vulnerable Node.js e-commerce application that implements all categories of the OWASP 10
Top 10. A Wazuh agent was deployed on this server and configured to monitor the application’s Apache access log. • Normal traffic generator. A Python script using the Selenium WebDriver library continuously submitted random legitimate requests (browse products, add to cart, register, login) to Juice Shop from a designated IP address throughout all experimental sessions. • Attack machines. One or more physically separate machines with distinct IP addresses performed attack sessions using the tools listed in Table 2.
Table 2: Attack tools used during dataset collection. Tool
Attack Type
Mode
Ref. sqlmap development team (2022)
SQLMAP
SQL Injection
Auto. + manual
Acunetix
Web Vulnerability Scanning / XSS
Automated
Acunetix (2022)
Burp Suite
Brute Force / Broken Auth.
Semi-automated
PortSwigger (2022)
XSS automation tool
Cross-Site Scripting
Automated
—
Gobuster
Web Scanning (dir. enum.)
Automated
Reeves and Mehlmauer (2022)
5.2 Ground-Truth Labelling For each attack session we recorded: (a) the source IP of the attacking machine; (b) the precise start and end timestamps of the attack. After data collection, Wazuh security events were exported from Elasticsearch using the Scan API. Each event was assigned a ground-truth label deterministically: events whose data.srcip matches a known attack IP and whose timestamp falls within a recorded attack window receive the corresponding attack class label; all remaining events are labelled NORMAL. This procedure avoids the ambiguity of manual event-by-event annotation and produces a clean ground truth tied to observable network identity.
5.3 Dataset Statistics Two-sided balancing strategy. SENSITIVE DATA EXPOSURE dominates the raw dataset with 26,558 instances (57.2%), while BROKEN AUTHENTICATION has only 300 instances (0.6%). We apply a two-sided balancing strategy to the training split: minority classes are oversampled to 1,250 instances each via SMOTE-NC, while the dominant class is randomly undersampled to 1,250 instances. The 95% reduction in SENSITIVE DATA EXPOSURE training instances is intentional: all six attack classes are operationally equally important to detect. The test set is evaluated at its natural class proportions (no balancing), providing an unbiased estimate of real-world performance. The raw dataset contains 46,454 security events with 43 fields each. Table 3 shows the class distribution before and after SMOTE-NC balancing of the training split. The dominance of SENSITIVE DATA EXPOSURE in the raw data is expected and reflects the operational behaviour of web scanning and SQL injection tools: a single SQLMAP or Acunetix session generates hundreds to thousands of HTTP requests, the 11
Selenium Traffic Generator (Normal Traffic)
SQLMAP (SQL Injection)
Legend Attack Machine
Acunetix (Web Scan / XSS)
Normal Requests
Normal Traffic
Burp Suite (Brute Force / Broken Auth.)
Attack Traffic
SIEM Component
OWASP Juice Shop (Vulnerable Web App) + Wazuh Agent
XSS Tool (Cross-Site Scripting)
Gobuster (Directory Enumeration)
Security Events (Syslog)
Wazuh Manager Cluster
Filebeat Elasticsearch Export (Scan API) Ground-Truth Labels (IP + timestamp matching)
Fig. 3: Testbed configuration used for dataset collection. Five physically separate attack machines run dedicated tools against the OWASP Juice Shop victim server, while a Selenium script generates concurrent normal traffic from a separate IP. The Wazuh Agent on the victim server forwards all security events to the Wazuh Manager cluster, which persists them in Elasticsearch via Filebeat. Ground-truth labels are assigned deterministically from IP identity and recorded attack-session timestamps.
Table 3: Class distribution of the raw dataset and the SMOTE-NC balanced training set.
Raw Dataset
Training Set (SMOTE-NC)
Count
%
Count
%
SENSITIVE DATA EXPOSURE SQL INJECTION NORMAL WEB SCAN BRUTE FORCE XSS BROKEN AUTHENTICATION
26,558
57.2
1,250
10.0
6,573
14.2
1,250
10.0
6,350
13.7
5,000
40.0
5,654
12.2
1,250
10.0
702
1.5
1,250
10.0
317
0.7
1,250
10.0
300
0.6
1,250
10.0
Total
46,454
100
12,500
100
Class
12
majority of which trigger Wazuh’s sensitive-data-exposure rules (e.g., responses that include stack traces, verbose error messages, or partial database query text). This is a realistic distribution for the attack tools used; it is not a data collection artefact.
5.4 Train / Validation / Test Split Events are partitioned 64%/16%/20% using stratified random sampling to preserve class proportions across all six attack classes (verified: the test set contains 19.9–20.0% of each class). SMOTE-NC was applied exclusively to the training split; no synthetic samples appear in the validation or test sets. Context feature construction and data split. Context features are computed on the complete event log sorted chronologically by timestamp, before any split. For each event ei , its context vector uses the N = 30 most recent prior events from the same source IP — strictly enforced by the dataset-construction code, which iterates backward through earlier-timestamped rows only (no future events can enter the window). Events are then partitioned into training (64%), validation (16%), and test (20%) using stratified random sampling, preserving class proportions across all six attack classes (test set contains 19.9–20.0% of each class, verified by inspection). Data isolation argument. Because context features are computed with strict temporal ordering, a test-set event’s context window may reference training-split events (those that occurred earlier in time from the same source IP). This does not constitute methodological leakage for two reasons. First, context features aggregate only behavioural signals — HTTP status-code distributions and MITRE ATT&CK technique frequencies — not class labels; no label from a training event can contaminate a test event’s feature vector. Second, this construction accurately mirrors the production deployment, where the Elasticsearch history query retrieves all chronologically prior events from a source IP regardless of when the model was trained. The reported test-set performance therefore represents a realistic estimate of operational accuracy under genuine deployment conditions. The seven source IPs correspond to the seven attack tools and traffic sources listed in Table 2 (one IP per row). Figure 4 summarises the full dataset construction and splitting pipeline.
6 Feature Engineering 6.1 Base Feature Set The base feature set consists of 16 fields extracted directly from the Wazuhnormalised security event, as listed in Table 4. These include HTTP-level metadata (data.protocol, data.id), rule properties (rule.level, rule.firedtimes, rule.id, rule.description, rule.groups), and compliance and threat-intelligence mappings (rule.pci dss, rule.hipaa, rule.nist 800 53, rule.gdpr, rule.tsc, rule.mitre.id, rule.frequency, rule.mail, agent.description). 13
46,454 Wazuh Security Events (exported from Elasticsearch)
Deterministic Ground-Truth Labelling by Source IP + Attack Timestamps
NORMAL 6,350 (13.7%)
Sensitive Data Exp. 26,558
Brute Force 702 7-Class Labelled Dataset — 46,454 events total SQL Inj. 6,573
Web Scan 5,654
XSS 317
Broken Auth. 300
Stratified Random Event-Level Split
Training Set 64 % — 29,731 events
Validation Set 16 % — 7,432 events (no SMOTE applied)
SMOTE-NC Balancing (training split only)
Test Set 20 % — 9,232 events (no SMOTE applied)
† evaluation only (no resampling) †
Balanced Training Set — 12,500 events NORMAL: 5,000 Each attack class: 1,250
Fig. 4: Dataset construction and splitting pipeline. Raw events are exported from Elasticsearch and assigned ground-truth labels deterministically from source IP and attack-session timestamps. The 46,454-event corpus is partitioned at session level into training (64%), validation (16%), and test (20%) splits. SMOTE-NC oversampling is applied exclusively to the training split; validation and test sets contain only real events at their natural class proportions.
6.2 Contextual History Features The context vector design is motivated by the principle that attack campaigns leave systematic traces across sequences of security events. Empirically, the ablation study (Table 8) demonstrates that removing all context features reduces macro F1 by 0.24–0.26 points across four gradient boosting algorithms. Figure 6 shows gain-based feature importance for the proposed hybrid, confirming that context features dominate Stage 1: 8 of the top-10 Stage 1 (LightGBM) features are context features, led by history.rule.firedtimes (gain = 20,573). Stage 2 (XGBoost) shows a more balanced split (5/10 context), reflecting that attack-type discrimination additionally requires protocol and rule-group information. The base feature set already includes rule.firedtimes for the current event ei , while the context vector includes hist.firedtimes = max(rule.firedtimes over N prior events). These capture distinct 14
Table 4: Base feature set extracted from Wazuh security events. Feature
Type
Description
data.protocol
Categorical
HTTP method (GET, POST, . . . )
data.id
Numerical
HTTP response status code
rule.firedtimes
Numerical
Times this rule fired in the last hour
rule.mail
Binary
Whether rule triggers an email alert
rule.level
Numerical
Rule severity (1–16)
rule.description
Categorical
Human-readable rule description
rule.groups
Categorical
Rule category tags
rule.pci dss
Categorical
PCI DSS requirement references
rule.tsc
Categorical
TSC control references
rule.nist 800 53
Categorical
NIST 800-53 control references
rule.gdpr
Categorical
GDPR article references
rule.mitre.id
Categorical
MITRE ATT&CK technique ID
rule.frequency
Numerical
Minimum event count before rule fires (integer threshold)
rule.hipaa
Categorical
HIPAA control references
agent.description
Categorical
Type of monitored agent
rule.id
Categorical
Numeric rule identifier
signals — instantaneous activation frequency vs. session peak — so the apparent redundancy is minimal. The high gain importance of hist.firedtimes confirms that the session-peak signal contributes discriminative information beyond the currentevent value alone. The base features retained in the context vector (rule.firedtimes, rule.description, data.protocol, data.id, rule.mitre.id) were selected to capture the dynamics of these high-importance fields over the preceding N events from the same source IP. Formally, let Hi = {ei−1 , ei−2 , . . . , ei−N } denote the set of at most N events preceding event ei that share the same data.srcip. We define twelve context features:
hist.firedtimes = maxe∈Hi e.rule.firedtimes. The peak activation count in the history window. Attackers who repeatedly probe the same endpoint cause the same rule to fire many times; this feature captures that escalation. hist.status.2xx, hist.status.3xx, hist.status.4xx, hist.status.5xx = |{e ∈ Hi : ⌊e.data.id/100⌋ = c}| for c ∈ {2, 3, 4, 5}. The count of responses in each HTTP status family. SQL injection and XSS tools generate characteristic distributions: many 4xx responses from rejected payloads, occasional 2xx responses when payloads succeed; web scanners produce a distinctive mix of 2xx and 3xx. 15
T1190, T1083, T1055, T1212, T1068, T1064, T1210 = |{e ∈ Hi : t ∈ e.rule.mitre.id}| for each MITRE technique identifier t. These seven techniques were selected a priori by inspecting the public Wazuh rule repository (version 4.x) for web application attack rules, prior to any data collection and without reference to the experimental dataset 1 . The selection criteria were: (1) the technique appears in at least ten distinct Wazuh rules in the web or appsec rule groups; (2) the technique is associated with web application attacks in the MITRE ATT&CK Enterprise matrix. The Wazuh rule repository was inspected at release tag v4.3.0 (commit wazuh/wazuh-ruleset, branch stable Wazuh, Inc. (2022)). The resulting seven identifiers are: T1190 (Exploit Public-Facing Application), T1083 (File and Directory Discovery), T1055 (Process Injection), T1212 (Exploitation for Credential Access), T1068 (Exploitation for Privilege Escalation), T1064 (Scripting), T1210 (Exploitation of Remote Services). Their cumulative counts across the history window create a technique-frequency profile that discriminates attack categories with complementary ATT&CK signatures. The complete feature vector fed to the classifier is thus the concatenation of the 16 base features and the 12 context features, yielding a 28-dimensional input, as illustrated in Figure 5. Events from source IP X (chronological order) ei−30
ei−29
History window Hi
···
ei−2
ei (current)
ei−1
(N = 30 events, same source IP)
Peak Rule Activation
HTTP Status Buckets
MITRE ATT&CK Counts
hist.firedtimes = maxe∈Hi e.rule.firedtimes
hist.status.2xx, 3xx, 4xx, 5xx count of responses per HTTP family
T1190 | T1083 | T1055 T1212 | T1068 | T1064 | T1210
(1 feature)
(4 features)
(7 features)
12 context features
16 Base Features (from current event ei directly):
rule.level, rule.firedtimes, rule.id, rule.groups, rule.mitre.id, data.protocol, data.id, . . .
28-Dimensional Feature Vector
(16 base + 12 context)
−→
Hybrid Cascade input
Fig. 5: Construction of the per-source-IP contextual feature vector. For each incoming event ei , the consumer retrieves the N =30 most recent prior events from the same source IP, forming the history window Hi . Three groups of context features are derived from this window: (1) the peak rule.firedtimes value, (2) counts of HTTP responses in each status-code family, and (3) cumulative counts of the seven most discriminative MITRE ATT&CK technique identifiers. These 12 context features are concatenated with the 16 base features extracted directly from ei , yielding the 28-dimensional input vector fed to the hybrid cascade classifier. 1
Wazuh rule repository v4.3.0: https://github.com/wazuh/wazuh-ruleset. Accessed November 2023.
16
Fig. 6: Gain-based feature importance for the proposed hybrid: Stage 1 (LightGBM, left) and Stage 2 (XGBoost, right). Green bars indicate MITRE ATT&CK-enriched context features; blue bars indicate base Wazuh metadata features. Stage 1 is dominated by context features (8 of top 10), confirming that behavioural history is the primary signal for binary attack detection. Stage 2 shows a more balanced split (5 of top 10 context), as attack-type discrimination additionally requires protocol and rulegroup information.
7 Hybrid Cascaded Classifier 7.1 Architecture Rationale A direct seven-class classifier must simultaneously separate NORMAL traffic from six attack classes while also distinguishing among attack classes whose base-feature distributions overlap substantially (e.g., SQL injection and sensitive data exposure share many Wazuh rule groups and MITRE identifiers). We instead decompose the problem into two stages:
• Stage 1 (Model 1). Binary classification: NORMAL vs. ATTACK. This model operates on the complete balanced dataset (NORMAL: 5,000; ATTACK: 7,500) and is optimised for high recall of the attack class to minimise missed detections. • Stage 2 (Model 2). Multi-class classification across the six attack categories. This model is trained only on attack-labelled events (all six classes balanced at 1,250 each via SMOTE-NC) and is specialised to discriminate between attack types rather than between normal and abnormal behaviour. The two-stage pipeline is illustrated in Figure 7.
7.2 Hyperparameter Optimisation Grid search was performed over the validation set for each model independently. Table 5 reports the hyperparameter grid and the selected configuration. 17
Enriched Event Vector (28-dimensional: 16 base + 12 context)
Stage 1 — LightGBM Binary Classifier NORMAL vs. ATTACK
NO
Classified: NORMAL
ATTACK? YES
Stage 2 — XGBoost Multi-class Classifier 6-Way Attack Category Classification
SQL Injection
Web Scanning
XSS
Brute Force
Broken Auth.
Sensitive Data Exp.
Write to wazuh-ai-hybrid-* index
Fig. 7: Two-stage hybrid cascade classification pipeline. Stage 1 (LightGBM) makes a binary NORMAL/ATTACK decision; events classified as NORMAL are written directly to the output index. Events flagged as ATTACK proceed to Stage 2 (XGBoost), which assigns one of six fine-grained attack categories. The two-stage design reduces inter-class confusion between normal traffic and low-frequency attack classes.
Table 5: Hyperparameter grid search and selected values for the hybrid cascade. Stage 1 uses LightGBM; Stage 2 uses XGBoost. Parameter
Selected Value
Search Range
n estimators learning rate max depth subsample
{300, 500, 800} {0.03, 0.06, 0.1} {6, 8, 10} {0.7, 0.85, 1.0}
colsample bytree reg lambda
{0.7, 0.85, 1.0} {1, 3, 7}
Stage 1 (LightGBM)
Stage 2 (XGBoost)
500
500
0.1
0.06
10
8
0.70
0.85
0.85
0.70
1
1
Several selected values sit at grid boundaries (LightGBM learning rate = 0.1, max depth = 10; XGBoost subsample = 0.85). To verify these are not under-searched, we examined validation F1 curves: the selected values showed stable or declining validation performance when manually extended beyond the grid boundary (LightGBM learning rate = 0.15 and max depth = 12 both reduced validation F1 by >0.005), confirming the grid boundary is not a limitation of the search range. Early stopping was applied with a patience of 50 rounds; the iteration count in Table 5 reflects the best validation loss checkpoint, not the search grid maximum. 18
7.3 Training Protocol Summary 1. Load the SMOTE-NC-balanced training CSV (history window N = 30). 2. Preprocess: fill missing categoricals with empty string; fill missing numerics with defaults; cast rule.id to string; flatten list-typed fields to their string representation. 3. For LightGBM and XGBoost: apply OrdinalEncoder to categorical features. For CatBoost (comparison only): define Pool objects with explicit cat features lists. 4. Train Stage 1 (LightGBM) on the full binary dataset (NORMAL + ATTACK). 5. Filter training set to attack-only records; train Stage 2 (XGBoost) on the six-class attack dataset. 6. Evaluate both models on the held-out test set and record per-class precision, recall, and F1 -score.
8 Experimental Results This section reports results from six experiments on the held-out test set (9,232 events, never seen during training or hyperparameter tuning). All macro-averaged F1 -scores are reported unless otherwise stated. Bootstrap 95% confidence intervals (B = 1,000 resamples over the held-out test set) are reported alongside point estimates in Table 7 to account for sampling variance, particularly in low-frequency classes (BROKEN AUTH: n = 60; XSS: n = 63). The proposed system is a hybrid cascade: LightGBM for Stage 1 (binary NORMAL/ATTACK) and XGBoost for Stage 2 (six-class attack categorisation), both trained on the context-enriched feature set with N = 30.
8.1 Algorithm Comparison Table 6 compares eight gradient boosting and baseline algorithms on the held-out test set with context features (N = 30). All sklearn-based algorithms use OrdinalEncoder for categorical features; CatBoost (Proposed) uses its native categorical handling. To isolate the encoding contribution, CatBoost (Ordinal) applies OrdinalEncoder with identical hyperparameters. All models were tuned with RandomizedSearchCV (n = 20, k = 3); CatBoost (comparison baseline) uses its built-in grid search. † CatBoost (Native) training time of 8,705 s vs. CatBoost (Ordinal) at 217 s reflects the overhead of the built-in grid search, which evaluates 24 parameter combinations on an 80/20 internal split with native categorical encoding (which recomputes category statistics per combination). CatBoost (Ordinal) reuses the best parameters found by the native search and trains once, explaining the 40× difference. LightGBM achieves the highest Stage 1 F1 (0.967) and XGBoost achieves the highest Stage 2 F1 (0.914) as evaluated on the held-out test set. The hybrid combination was selected based on validation-set performance (each algorithm was evaluated on the 16% internal validation split during hyperparameter search); the test set was used only once for the final evaluation. Combining them in the hybrid cascade yields the best overall system. The two gradient boosting baselines (Logistic Regression and Decision Tree) confirm that ensemble tree methods are the appropriate model family for this task. 19
Table 6: Algorithm comparison on the held-out test set (context-enriched features, N = 30, macro-averaged metrics). Best result per metric in bold. Algorithm
Enc.
Stage 1 (Binary)
Stage 2 (Multi-class)
P
R
F1
P
R
F1
Time (s)
CatBoost (Proposed) CatBoost (Ordinal)
Native Ordinal
0.92 0.93
0.98 0.98
0.947 0.956
0.81 0.85
0.97 0.98
0.876 0.902
8705† 217
Random Forest Extrem. Rand. Trees XGBoost LightGBM Logistic Regression Decision Tree
Ordinal Ordinal Ordinal Ordinal Ordinal Ordinal
0.94 0.93 0.94 0.95 0.77 0.91
0.98 0.98 0.98 0.99 0.85 0.97
0.961 0.954 0.961 0.967 0.800 0.936
0.85 0.85 0.87 0.84 0.61 0.75
0.98 0.98 0.98 0.98 0.84 0.94
0.905 0.901 0.914 0.898 0.670 0.817
20 13 206 63 33 1
Hybrid (LGB S1 + XGB S2)
Ordinal
0.95
0.99
0.967
0.87
0.98
0.914
269‡
† ‡
CatBoost native uses built-in grid search; others use RandomizedSearchCV (n=20, k=3). Hybrid time = LightGBM Stage 1 (63 s) + XGBoost Stage 2 (206 s).
Three diagrams illustrating the algorithm comparison are shown in Figures 8, 9, and 10.
Fig. 8: Algorithm comparison: macro F1 -scores for Stage 1 (binary) and Stage 2 (multi-class) across all eight algorithms. The hybrid model (LightGBM S1 + XGBoost S2) achieves the best combined performance. ⋆ = Proposed system.
The confidence intervals confirm that the headline results are not artefacts of sampling noise. The Stage 1 CI for the hybrid ([0.962, 0.973]) is entirely above 0.95, confirming robustly superior binary detection. Stage 2 intervals are wider, as expected 20
Fig. 9: Radar chart of the full metrics profile (Stage-1 Precision, Recall, F1 ; Stage-2 Precision, Recall, F1 ) for each algorithm. The hybrid model (thick solid line) dominates across all six metrics.
Fig. 10: Efficiency vs. accuracy trade-off: macro F1 -score against training time (seconds) for all eight algorithms (⋆ = Proposed hybrid). The hybrid cascade achieves the highest F1 at a total training time comparable to XGBoost alone.
21
Table 7: Bootstrap 95% confidence intervals (B = 1000 resamples) for macro F1 -scores on the held-out test set (n = 9,232 events). Wide Stage 2 intervals reflect small minority-class test sizes (60 events for BROKEN AUTH., 63 for XSS). Algorithms marked “—” did not store predictions at checkpoint time; their point estimates are reported from Table 6. Algorithm
S1 F1
S1 95% CI
S2 F1
S2 95% CI
CatBoost (Proposed) XGBoost LightGBM Hybrid (LightGBM+XGBoost)
0.947 0.961 0.967 0.968
[0.940, 0.953] [0.956, 0.967] [0.962, 0.973] [0.962, 0.973]
0.879 0.905 0.880 0.906
[0.862, 0.894] [0.889, 0.918] [0.864, 0.895] [0.889, 0.921]
given that minority classes (BROKEN AUTH.: 60 test events, XSS: 63 test events) introduce more variance; the lower bound of 0.889 still substantially exceeds the 0.561–0.670 without-context baseline. These CIs address the concern that per-class F1 differences of ±0.02 between models are within sampling noise: the Hybrid vs. next-best XGBoost S2 difference (0.906 vs. 0.905) is indeed within noise, but the headline improvement over the without-context baseline (+0.32) far exceeds the CI width.
8.2 Impact of Contextual Features Table 8 reports the impact of adding the 12 context features to the 16 base features across five algorithms spanning two model families (gradient boosting and linear). Without context, the four gradient boosting algorithms converge to ≈0.705 Stage 1 F1 ; Logistic Regression starts slightly higher at 0.771 owing to its stronger regularised boundary on the base features alone. With context features, gradient boosting models improve substantially (+0.24 to +0.26 Stage 1, +0.30 to +0.35 Stage 2), while Logistic Regression improves more modestly (+0.03 Stage 1, +0.19 Stage 2).
Table 8: Impact of context-enriched features (N = 30) on five algorithms across two model families. Base uses 16 base features only; +Context adds 12 MITRE ATT&CK-enriched context features. All algorithms use OrdinalEncoding and fixed best hyperparameters. Stage 1 (Binary)
Algorithm Base
+Context
Stage 2 (Multi-class) ∆
Base
+Context
∆
Gradient boosting (tree ensembles) CatBoost 0.706 0.947 XGBoost 0.706 0.961 LightGBM 0.705 0.967 Random Forest 0.704 0.961
+0.241 +0.255 +0.262 +0.258
0.561 0.586 0.596 0.552
0.876 0.914 0.898 0.905
+0.315 +0.327 +0.301 +0.353
Linear model (non-GBM baseline) Logistic Regression 0.771 0.800
+0.029
0.478
0.670
+0.192
22
The near-identical without-context scores (≈0.705) across all four gradient boosting algorithms reveals that the base feature set saturates at this level — likely because one or a few features (rule.id, rule.description, rule.groups) carry the bulk of the discriminative signal available in the base feature set, and the ensembles all reach the same ceiling defined by that signal. Adding the context vector unlocks discriminative power that base features alone cannot express — this is the primary finding, and is algorithm-agnostic within the gradient boosting family. Logistic Regression also improves with context but substantially less (+0.03 Stage 1, +0.19 Stage 2 vs +0.25/+0.32 for gradient boosting), suggesting that the MITRE ATT&CK-enriched context vector encodes non-linear feature interactions that tree ensemble methods exploit more effectively than linear classifiers — a finding consistent with the gradient boosting literature on tabular data Grinsztajn et al. (2022). The central empirical finding is that context features substantially improve attack detection across all tested algorithm families ; gradient boosting models benefit most.
8.3 Cascade vs. Flat Classifier Table 9 compares the proposed hybrid cascade (LightGBM Stage 1 + XGBoost Stage 2) against a single flat seven-class LightGBM trained with the same Stage 1 hyperparameters and context features. This directly addresses the question of whether the cascaded architecture adds value over a single-stage classifier using the same underlying algorithm.
Table 9: Hybrid cascade (LightGBM Stage 1 + XGBoost Stage 2) vs. flat seven-class LightGBM: macro F1 and securityoriented metrics (same Stage 1 hyperparameters, context features, N = 30). Attack Recall is the fraction of true attacks detected; Missed Attacks counts true attacks labelled as NORMAL. Metric
Flat LightGBM
Hybrid Cascade
0.926 0.982 0.885 143 12
0.906 0.984 0.855 131 12
Macro F1 (academic) Attack Recall Attack Precision Missed Attacks (false negatives) False Alarms (FP on NORMAL)
Bold = operationally superior value.
The cascaded design achieves slightly lower macro F1 (0.906 vs. 0.926) but reduces missed attacks from 143 to 131 (−8.4%) while maintaining identical false alarms (12 each). Attack recall improves marginally (0.984 vs. 0.982). In security operations, the cost of a missed attack vastly exceeds the cost of a false alarm Sommer and Paxson (2010); the cascade is therefore the operationally superior design despite the aggregate F1 trade-off. To disambiguate whether the performance difference stems from the cascaded architecture or from using XGBoost in Stage 2, Table 11 adds a flat XGBoost 23
Table 10: Per-class F1 : flat LightGBM vs. hybrid cascade (LightGBM Stage 1 + XGBoost Stage 2), context features, N = 30. Recall values show the cascade achieves higher or equal recall on all attack classes. Class
Flat LightGBM
Hybrid Cascade
F1
Recall
F1
Recall
BROKEN AUTH. BRUTE FORCE NORMAL SENSITIVE DATA EXP. SQL INJECTION WEB SCAN XSS
0.896 0.959 0.939 0.961 0.987 0.896 0.844
1.000 1.000 0.990 0.934 0.976 0.956 0.984
0.800 0.966 0.944 0.956 0.993 0.881 0.800
1.000 1.000 0.990 0.924 0.989 0.948 0.984
Macro avg
0.926
0.982
0.906
0.984
seven-class baseline. The hybrid cascade achieves 0.984 attack recall and 131 missed attacks — fewer than both flat LightGBM (143 missed) and flat XGBoost (174 missed) — confirming that the cascaded architecture contributes independently of algorithm choice.
Table 11: Three-way architecture comparison: flat LightGBM, flat XGBoost, and the proposed hybrid cascade (context features, N = 30). Flat models use the same best hyperparameters as their respective cascade stages. The hybrid cascade achieves the lowest missed-attack count despite similar macro F1 to flat XGBoost. Model Flat LightGBM (Stage 1 params) Flat XGBoost (Stage 2 params) Hybrid Cascade (proposed)
Macro F1
Attack Recall
Missed Attacks
0.926 0.905 0.906
0.982 0.978 0.984
143 174 131
The confusion matrices for both stages are shown in Figure 11. Note on SENSITIVE DATA EXPOSURE taxonomy. At 57.2% of the raw event log, SENSITIVE DATA EXPOSURE represents Wazuh sensitive-data rule activations triggered as side-effects of SQLMAP and Acunetix probes, not a discrete OWASP attack class on the same axis as SQL INJECTION. A cleaner formulation would drop it or reframe it as a campaign stage; we retain it for consistency with the Wazuh rule taxonomy. Its dominance inflates Stage 2 weighted-average metrics and its F1 score (0.956 / 0.961 flat vs. cascade) should be interpreted in this context. 24
Fig. 11: Confusion matrices for the proposed hybrid cascade (LightGBM Stage 1 and XGBoost Stage 2, context features, N = 30). Left: Stage 1 binary classifier. Right: Stage 2 multi-class attack classifier (cell values show row-normalised percentage and absolute count). Off-diagonal entries in Stage 2 are concentrated between BROKEN AUTHENTICATION and SENSITIVE DATA EXPOSURE, which share MITRE identifiers.
8.4 Ablation Study: Context Window Size Figure 12 plots macro F1 against context window size N for three algorithms (CatBoost, LightGBM, XGBoost), each trained with fixed best hyperparameters to isolate the effect of N . The multi-algorithm ablation demonstrates that the N = 30 practical operating point is not an artefact of any particular algorithm but an intrinsic property of the attack behaviour captured in the dataset: the largest gains occur between N = 3 and N = 15, with diminishing returns thereafter. We adopt N = 30 as the production default.
8.5 Comparison with the Wazuh Rule Engine Table 12 compares the Wazuh native rule engine against the AI module across all six attack classes. We use a strict definition of “rule-identified”: an event is counted only if its rule.groups, rule.description, or rule.mitre.id fields explicitly name the attack category. This intentionally excludes generic Wazuh alerts (e.g., “Multiple web server 400 error codes”, “Suspicious URL access”) that fire on attack traffic but do not identify the attack type—such alerts are operationally insufficient as they prevent category-appropriate incident response. The strict metric therefore measures actionable, categorised detection rather than mere alert generation. Wazuh’s rule engine achieves 0% categorised detection on Brute Force and Broken Authentication: Wazuh does raise generic alerts for these events (e.g., multiple failedlogin rules), but those alerts do not identify the attack category, preventing categoryspecific incident response. The AI module correctly categorises them at 100% and 98.3% respectively. Across all six classes, the average AI detection rate is 95.8%, compared to 5.8% for the rule engine. 25
Stage 1:Binary (NORMAL vs. ATTACK)
Stage 2:Multi-class (6 attack types)
1
Macro F1 -score
Macro F1 -score
0.95 0.9 0.85 0.8 0.75 0.7 3 5 7 1012 15 20 25 30 Context window size N
35
3 5 7 1012 15 20 25 30 Context window size N
CatBoost LightGBM XGBoost Upper bound
Fig. 12: Ablation study: macro F1 vs. context window size N for Stage 1 (binary) and Stage 2 (multi-class). All three algorithms continue improving through N = 35 (LightGBM S1: 0.967 → 0.977; XGBoost S1: 0.962 → 0.975; CatBoost S2: 0.880 → 0.894), with the largest gains between N = 3 and N = 15. N = 30 is adopted as the practical operating point balancing accuracy against Elasticsearch query latency.
Table 12: Wazuh rule engine vs. Smart-SIEM AI module. Wazuh (Cat.) counts events whose rule fields explicitly name the attack category (strict metric). Wazuh (Any) counts events triggering any Wazuh rule at level ≥3. Wazuh raises generic alerts on many attack events but cannot assign attack categories, preventing category-specific incident response. AI% is computed over the held-out test set (Test Events column). Class
Raw Events
Wazuh Cat.%
Wazuh Any%
Test Events
AI Det.
AI%
SQL INJECTION XSS WEB SCAN BRUTE FORCE BROKEN AUTH. SENSITIVE DATA EXP.
6,573 317 5,654 702 300 26,558
7.4 17.0 1.3 0.0 0.0 8.5
10.5 36.9 34.3 85.8 19.3 9.8
1,315 63 1,131 140 60 5,312
1,276 62 1,036 140 59 4,748
97.0 98.4 91.6 100.0 98.3 89.4
26
35
8.6 Self-Adaptive Retraining Table 13 demonstrates the self-adaptive retraining mechanism under a controlled concept drift scenario. The initial model is trained on events from three known attack types (SQL Injection, XSS, Web Scanning). Phase 2 introduces three previously unseen attack types (Brute Force, Broken Authentication, Sensitive Data Exposure), simulating the emergence of new threat patterns in production.
Table 13: Self-adaptive retraining simulation (extended). Phase 1 trains on NORMAL + SQL Injection + XSS + Web Scanning. Phase 2 introduces three previously unseen attack types (Brute Force, Broken Authentication, Sensitive Data Exposure). Phase 2-only retraining uses drift data alone; Phase 1+2 retraining uses the combined corpus, representing the production-intended operational protocol. Data Segment
Initial Model
Phase 2 Only
Phase 1+2 Combined
Phase 1 (known attacks) Phase 2 (concept drift) Phase 3 (all attacks)
0.905 0.465 0.596
— 0.779 0.695
0.817 0.784 0.811
Recovery (Phase 3)
—
+0.099
+0.215
The F1 score drops from 0.905 to 0.465 when the model encounters the unseen attack types in Phase 2 — well below the configured 90% threshold — triggering the retraining procedure. After retraining on Phase 2 events with analyst-provided labels, the model recovers to 0.695 on the fully held-out Phase 3 set, a gain of +0.099 (Phase 2-only column of Table 13). The production-intended protocol — retraining on the combined Phase 1+Phase 2 corpus (41,737 events) — recovers Phase 3 F1 to 0.811 (+0.215 over the initial model), demonstrating that preserving knowledge of originally-known attack classes substantially improves recovery. One consequence of the combined protocol is a modest degradation on Phase 1 data: the combined model achieves F1 = 0.817 on Phase 1 vs. the original 0.905 — a −0.09 drop. This reflects the expected effect of training on a more diverse corpus: balancing six attack classes rather than three reduces the effective training weight on the three originally-known classes. Production operators can mitigate this through classweighted loss functions, experience replay (reserving a fixed proportion of Phase 1 samples in each retraining batch), or by monitoring per-class F1 alongside the aggregate threshold to detect class-specific regressions before they become operationally significant.The incomplete recovery to 0.695 (vs. the original 0.905 on known attacks only) reflects two factors. First, Phase 3 contains all six attack types while Phase 1 27
contained only three; the 0.905 baseline was measured on a simpler distribution. Second, Phase 2 is dominated by SENSITIVE DATA EXPOSURE (93.2% of Phase 2 events), limiting balanced exposure to Brute Force and Broken Authentication during retraining. In production, the operator would retrain on a combined Phase 1+Phase 2 corpus to preserve knowledge of originally-known attack classes. The simulation demonstrates the mechanism functions correctly: drift is detected, retraining triggers, and accuracy recovers. The retraining simulation figure is shown in Figure 13.
Fig. 13: Self-adaptive retraining simulation. The initial model encounters Phase 2 (three previously unseen attack types), causing F1 to drop to 0.465 — below the 90% threshold — and triggering retraining. Phase 2-only retraining recovers Phase 3 to 0.695 (+0.099). The production-intended Phase 1+2 combined protocol recovers Phase 3 to 0.811 (+0.215), substantially stronger recovery (see Table 13).
8.7 System Performance Characteristics The consumer processes events at a rate sufficient for near-real-time operation. Kafka decouples the Wazuh event stream from the classification workers, absorbing burst traffic during active attack sessions. Based on informal measurements during testbed operation, LightGBM and XGBoost prediction latency is sub-millisecond for a single event; Elasticsearch context retrieval (the N = 30 history query against an unloaded local cluster) adds approximately 15–50 ms per event, constituting the dominant pipeline latency. These figures are informal estimates collected during testbed operation under low concurrency; they are provided for orientation only. A reproducible micro-benchmark (event-level pipeline latency at p50/p95/p99, with a realisticallysized Elasticsearch corpus) is needed for production deployment sizing and remains future work. 28
9 Discussion 9.1 Why Behavioural Context Matters for SIEM Classification The magnitude of the performance gap between base-only and context-enriched models (Table 8) provides strong empirical support for a principle articulated by Sommer and Paxson (2010) but rarely quantified in the SIEM literature: security events do not occur in isolation. An HTTP 200 response to a request for /api/user is entirely ambiguous in isolation; the same response following a history window that includes 400 prior activations of T1190 (Exploit Public-Facing Application) and 150 HTTP 4xx responses is almost certainly part of an active SQL injection campaign. The context vector encodes exactly this type of escalating behavioural signal, transforming a stateless event classifier into a session-aware detector aligned with the session-based intuition of anomaly detection Chandola et al. (2009) and multi-stage attack modelling Nisioti et al. (2018). This result also reinforces the core limitation of purely rule-based correlation engines identified in Section 2: rules evaluate each event without access to its behavioural antecedents, and the average +0.25 (Stage 1) and +0.32 (Stage 2) F1 improvement for gradient boosting algorithms from context features (Logistic Regression also improves, +0.03/+0.19, but substantially less) achieved by adding context features quantifies precisely how much signal is discarded by stateless evaluation Garcia-Teodoro et al. (2009); Bhatt et al. (2014).
9.2 The Role of MITRE ATT&CK Technique Counts The seven MITRE technique frequency features collectively contribute a meaningful portion of the model’s discriminative power, as confirmed by SHAP-based feature attribution Lundberg and Lee (2017) and evidenced by the reduction in cross-class confusion visible in the per-class F1 results (Table 10). Different attack categories exhibit distinct technique accumulation profiles: SQL injection accumulates T1190 (Exploit Public-Facing Application) rapidly, while web scanning accumulates T1083 (File and Directory Discovery). This aligns with the threat intelligence literature’s observation that ATT&CK techniques provide a stable, attack-campaign-level abstraction that persists across diverse tooling implementations Husari et al. (2017); Milajerdi et al. (2019); Xiong and Lagerström (2019). The finding suggests that ATT&CK-enriched contextual profiling is a promising direction for multi-source SIEM correlation beyond the web application domain studied here.
9.3 Comparison with Prior ML-SIEM Approaches Absence of a direct numerical comparison. To the best of our knowledge as of the submission date, no prior published work operates on Wazuh-format SIEM events with per-IP MITRE ATT&CK contextual history, though related work using MITRE-tagged log data in other formats exists Milajerdi et al. (2019); Xiong and Lagerström (2019); a direct numerical comparison remains methodologically impossible: re-implementing a prior method on our dataset would require that method to have a Wazuh-compatible data ingestion layer, while applying our context features to a public dataset (NSL-KDD, CICIDS-2017) would require those datasets to contain 29
per-event MITRE technique identifiers and per-IP history sequences — fields that are absent from all public IDS datasets we are aware of. We provide two illustrative (but methodologically limited) reference points to contextualise performance. Important caveat: cross-dataset F1 comparisons across datasets with different feature spaces, class counts, and collection protocols are inherently limited; the following figures are provided for orientation only and should not be interpreted as direct performance equivalences. First, our without-context Stage 1 F1 of ≈0.705 is in the same range as macro F1 values reported for supervised classifiers on NSL-KDD (0.70–0.78 Tavallaee et al. (2009)) and CICIDS-2017 (0.72–0.85 Sharafaldin et al. (2018)); this illustrates that the base feature set is comparably expressive to prior IDS benchmarks, though the datasets differ fundamentally. Second, Veeramachaneni et al. (2016) report ≈85% detection on generic log streams; our system achieves 95.8% average AI coverage, though these rates measure different threat models and cannot be directly compared. Compared to the closest prior work, Smart-SIEM differs in three substantive ways. First, whereas most ML-based SIEM augmentation systems operate on packetlevel network flow datasets Tavallaee et al. (2009); Sharafaldin et al. (2018), this work operates natively on Wazuh-normalised security events, preserving the compliance mappings (PCI DSS, HIPAA, NIST 800-53), MITRE technique identifiers, and rule metadata that flow datasets do not contain. Second, prior SIEM-augmentation proposals either operate without contextual history Sarker et al. (2020) or require sustained analyst labelling to function Veeramachaneni et al. (2016); the proposed framework automatically constructs contextual features from Elasticsearch query results and requires labelling only for knowledge-base construction, not for inference. Third, the self-adaptive retraining mechanism directly addresses concept drift—a welldocumented challenge in operational security classifiers Yang et al. (2021); Pendlebury et al. (2019); Gama et al. (2014)—by monitoring accuracy against the knowledge base and triggering retraining when it degrades, rather than relying on periodic scheduled retraining that may lag or overshoot the drift event.
9.4 Limitations and Threats to Validity This study has four principal limitations that future work should address. 1. Single-session-per-class and label-by-proxy confound. Our testbed assigns one dedicated source IP per attack class. Because labels are deterministic by IP, the context features (MITRE technique counts, HTTP status-code histograms computed over prior events from the same IP) are statistical proxies for the class label. The current design cannot distinguish between (a) learning a generalizable behavioural fingerprint of attack campaigns, and (b) learning the per-IP identity of the testbed. Leave-one-IP-out (or leave-one-tool-out) evaluation is the standard remedy, but requires multiple independent sessions per attack class — a multisession testbed remains a priority for future work. The reported F1 values should be interpreted as an upper bound on generalisation performance. 2. Context feature normalisation and cold-start bias. The context features (hist.status.2xx–5xx and the seven MITRE technique counts) are raw counts over the N = 30 history window. For source IPs with fewer than N prior events 30
(e.g., the first events of a new attack campaign), these counts are systematically smaller, biasing the classifier toward labelling early-session events as normal. Future work should normalise by the actual number of retrieved history events to eliminate this cold-start artefact, or explicitly evaluate performance on events with fewer than N prior events. 3. Single-application scope. All data were collected from a single web application (OWASP Juice Shop) on a single server. While Juice Shop covers the OWASP Top 10 OWASP Foundation (2021a) and is widely used in security research, the Wazuh rule groups and MITRE identifiers it triggers may not fully represent other application stacks (Java EE, .NET, mobile back-ends) or network-layer attack surfaces Stuttard and Pinto (2011). 4. Limited attack diversity. The dataset covers six attack categories generated by five tools. Real-world SIEM environments must handle a much wider range of attack types, including insider threats, advanced persistent threats (APTs), supply-chain compromises, and novel zero-day exploits for which no labelled data exist Apruzzese et al. (2018). The severity of this limitation is partially mitigated by the selfadaptive retraining mechanism, but a broader multi-tool, multi-application testbed remains a priority for future work. 5. Controlled testbed vs. production traffic. Normal traffic is generated by a Selenium script, which does not capture the distributional richness of real human user behaviour at scale Sommer and Paxson (2010). Transfer performance to a production environment will require incremental retraining on analyst-labelled production events using the self-adaptive mechanism. 6. Model interpretability. Gradient-boosted classifiers remain opaque relative to rule-based engines, which is a non-trivial barrier to adoption in regulated industries where detection decisions must be explainable Bhatt et al. (2014). The gainbased feature importance analysis (Figure 6) provides partial interpretability, but integration with formal explanation interfaces remains future work. 7. Deployment compliance overhead. Configuring the system requires bulk modification of Wazuh rule levels to route events to the classification pipeline (Section 4). In PCI DSS or HIPAA-regulated environments this constitutes a change management event that may require compliance re-certification, representing a non-trivial deployment overhead for regulated operators.
9.5 Analyst Alert Load and Confidence-Based Routing A practical concern in deploying any AI-augmented SIEM is analyst alert fatigue : when the volume of generated alerts exceeds the capacity of the security operations centre (SOC) to review them, analysts become desensitised and critical alerts may be overlooked Julisch (2003); Bhatt et al. (2014). Smart-SIEM inherits this challenge because the hybrid cascade classifies every event that Stage 1 flags as ATTACK, producing a categorised alert regardless of how confidently the model reached that decision. A principled extension that addresses this concern without modifying the experimental design is confidence-based selective routing, known in the active learning literature as uncertainty sampling Settles (2012). Both LightGBM (Stage 1) and 31
XGBoost (Stage 2) expose predict proba outputs, so a per-event confidence score pmax = maxc P̂ (y =c | x) is available at inference time without any architectural change. Two separate routing decisions can be derived from this score: 1. Dashboard visibility (attack alerting). All events classified as ATTACK — regardless of confidence — are written to the wazuh-ai-hybrid-* index and surfaced in the Kibana dashboard. This ensures that no detected attack is hidden from the SOC. 2. Knowledge-base labelling queue (analyst workload). Only events whose Stage 1 or Stage 2 confidence falls below a configurable threshold τ are forwarded to the analyst labelling interface. High-confidence predictions (pmax ≥ τ ) are added automatically to the knowledge base, reducing repetitive labelling of straightforward cases. This two-pipeline design is critical: it reduces analyst labelling workload without reducing alert visibility, directly addressing the operational concern while preserving full detection coverage. Low-confidence NORMAL predictions deserve particular attention — an event classified as normal with low confidence represents a potential false negative that would otherwise be silently discarded, and routing such events to the analyst queue captures the highest-risk ambiguous cases before they are lost. One important caveat is that gradient boosting classifiers are not natively probability-calibrated Niculescu-Mizil and Caruana (2005): raw predict proba scores may be systematically over- or under-confident relative to the true posterior. Applying post-hoc calibration (e.g., Platt scaling or isotonic regression) before thresholding is therefore a prerequisite for production deployment of confidence-based routing, ensuring that pmax ≥ τ reliably corresponds to high empirical accuracy.
10 Conclusion and Future Work This paper presented Smart-SIEM, a modular AI enhancement for the Wazuh open-source SIEM platform that addresses the core limitation of rule-based event correlation: the inability to exploit the behavioural context accumulated across a sequence of related security events Garcia-Teodoro et al. (2009); Bhatt et al. (2014). By constructing a per-source-IP context vector that summarises HTTP responsecode distributions and MITRE ATT&CK technique frequency profiles Strom et al. (2018) from the preceding N = 30 security events, and feeding this vector into a hybrid cascade combining LightGBM for binary attack detection Ke et al. (2017) and XGBoost for attack classification Chen and Guestrin (2016), we achieve macro F1 scores of 0.967 (binary) and 0.914 (six-class) on a purpose-built SIEM event dataset. A controlled multi-algorithm comparison demonstrates that without context features all tested gradient boosting algorithms converge to approximately 0.705 F1 , rising to 0.947–0.967 with context — an average improvement of +0.254 Stage 1 and +0.324 Stage 2, confirming that the context vector is the primary contribution rather than any specific algorithm choice. These results represent an upper bound on generalisation performance given the single-session-per-class testbed design (Section 9.4). The AI module detects an average of 95.8% of attack events across all six classes, compared 32
to 5.8% for Wazuh’s native rule engine; Brute Force and Broken Authentication, which Wazuh misses entirely (0%), are detected at 100% and 98.3% respectively. The selfadaptive retraining mechanism directly addresses the concept-drift challenge Gama et al. (2014); Lu et al. (2019): when three previously unseen attack types are introduced (simulating production drift), the F1 drops from 0.905 to 0.465, triggering retraining; partial recovery to 0.695 (+0.099) with Phase 2-only retraining; the combined Phase 1+2 protocol recovers to 0.814. Future directions include: (1) extending the testbed to cover non-web attack surfaces (network scans, lateral movement, data exfiltration via DNS) to assess whether ATT&CK-enriched context generalises across domains; (2) evaluating deep sequence models (LSTM, Transformer) as alternatives to the fixed-width aggregation context vector Ferrag et al. (2020); Vinayakumar et al. (2019); (3) building a public benchmark dataset of Wazuh-format SIEM events to support reproducible ML-SIEM research, addressing the gap highlighted by Ring et al. (2019); and (4) formalising the self-adaptive retraining procedure with theoretical guarantees on convergence under concept drift Yang et al. (2021); Pendlebury et al. (2019); and (5) implementing confidence-based selective routing (Section 9.5), in which only events whose posterior probability falls below a configurable threshold are forwarded to the analyst labelling queue while high-confidence predictions are added to the knowledge base automatically — a design that directly addresses analyst alert fatigue Julisch (2003) and formalises the self-adaptive retraining loop as an uncertainty-guided active learning system Settles (2012).
Acknowledgements. This work was carried out at the Higher Institute for Applied Sciences and Technology (HIAST), Damascus, Syria. The authors thank the HIAST faculty for providing the computational resources used in this research.
Declarations Funding Statement. The authors received no specific funding for this work. Conflicts of Interest. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Availability of data and materials. The labelled Wazuh security event dataset (46,454 records) used in this study and the Smart-SIEM AI module source code are both available from the corresponding author upon reasonable request. The dataset contains security events collected from a controlled testbed and does not include any personally identifiable information. Code availability. The source code of the Smart-SIEM module is available from the corresponding author upon reasonable request.
References Acunetix (2022) Acunetix web vulnerability scanner. URL https://www.acunetix.com, accessed: 2022-07-01 33
Agrawal K, Makwana H (2015) A study on critical capabilities for security information and event management. International Journal of Science and Research (IJSR) 4(7):1893–1896 Apruzzese G, Colajanni M, Ferretti L, et al (2018) On the effectiveness of machine and deep learning for cyber security. In: 2018 10th International Conference on Cyber Conflict (CyCon), pp 371–390, https://doi.org/10.23919/CYCON.2018.8405026 Bhatt SN, Manadhata PK, Zomlot L (2014) The operational role of security information and event management systems. IEEE Security & Privacy 12(5):35–41. https://doi.org/10.1109/MSP.2014.103 Breiman L (2001) Random forests. Machine Learning 45(1):5–32. https://doi.org/10. 1023/A:1010933404324 Buczak AL, Guven E (2016) A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Communications Surveys & Tutorials 18(2):1153–1176. https://doi.org/10.1109/COMST.2015.2494502 Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: A survey. ACM Computing Surveys 41(3):15:1–15:58. https://doi.org/10.1145/1541880.1541882 Chawla NV, Bowyer KW, Hall LO, et al (2002) SMOTE: Synthetic minority oversampling technique. Journal of Artificial Intelligence Research 16:321–357. https: //doi.org/10.1613/jair.953 Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 785–794, https://doi.org/10.1145/2939672.2939785 Chuvakin AA, Schmidt KJ, Phillips C (2012) Logging and Log Management: The Authoritative Guide to Understanding the Concepts Surrounding Logging and Log Management. Syngress Creech G, Hu J (2014) A semantic approach to host-based intrusion detection systems using contiguous and discontiguous system call patterns. IEEE Transactions on Computers 63(4):807–819. https://doi.org/10.1109/TC.2013.13 Denning DE (1987) An intrusion-detection model. IEEE Transactions on Software Engineering 13(2):222–232. https://doi.org/10.1109/TSE.1987.232894 Dorogush AV, Ershov V, Gulin A (2018) CatBoost: Gradient boosting with categorical features support. arXiv preprint arXiv:181011363 Fernández A, Garcı́a S, Herrera F, et al (2018) SMOTE for learning from imbalanced data: Progress and challenges, marking the 15-year anniversary. Journal of Artificial Intelligence Research 61:863–905. https://doi.org/10.1613/jair.1.11192
34
Ferrag MA, Maglaras L, Moschoyiannis S, et al (2020) Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study. Journal of Information Security and Applications 50:102419. https://doi.org/10.1016/j.jisa.2019. 102419 Gama J, Žliobaitė I, Bifet A, et al (2014) A survey on concept drift adaptation. ACM Computing Surveys 46(4):44:1–44:37. https://doi.org/10.1145/2523813 Garcia-Teodoro P, Diaz-Verdejo J, Maciá-Fernández G, et al (2009) Anomaly-based network intrusion detection: Techniques, systems and challenges. Computers & Security 28(1–2):18–28. https://doi.org/10.1016/j.cose.2008.08.003 González-Granadillo G, González-Zarzosa S, Diaz R (2021) Security information and event management (SIEM): Analysis, trends, and usage in critical infrastructures. Sensors 21(14):4759. https://doi.org/10.3390/s21144759 Grinsztajn L, Oyallon E, Varoquaux G (2022) Why tree-based models still outperform deep learning on tabular data. In: Advances in Neural Information Processing Systems, vol 35. Curran Associates, Inc., pp 507–520 Grossman J (2007) Cross-site scripting worms and viruses: The impending threat and the best defense. Tech. rep., WhiteHat Security Guo H, Li Y, Shang J, et al (2017) Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications 73:220–239. https: //doi.org/10.1016/j.eswa.2016.12.035 Halfond WGJ, Viegas J, Orso A (2006) A classification of SQL-injection attacks and countermeasures. In: Proceedings of the International Symposium on Secure Software Engineering (ISSSE), pp 13–15 He H, Garcia EA (2009) Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering 21(9):1263–1284. https://doi.org/10.1109/TKDE.2008. 239 Husari G, Al-Shaer E, Ahmed M, et al (2017) TTPDrill: Automatic and accurate extraction of threat actions from unstructured text of CTI sources. In: Proceedings of the 33rd Annual Computer Security Applications Conference (ACSAC), pp 103– 115, https://doi.org/10.1145/3134600.3134646 Julisch K (2003) Clustering intrusion detection alarms to support root cause analysis. ACM Transactions on Information and System Security 6(4):443–471. https://doi. org/10.1145/950191.950192 Ke G, Meng Q, Finley T, et al (2017) LightGBM: A highly efficient gradient boosting decision tree. In: Advances in Neural Information Processing Systems (NeurIPS)
35
Kimminich B (2021) OWASP Juice Shop: Probably the most modern and sophisticated insecure web application. URL https://owasp.org/www-project-juice-shop/, accessed: 2022-07-01 Kwon D, Kim H, Kim J, et al (2019) A survey of deep learning-based network anomaly detection. Cluster Computing 22:949–961. https://doi.org/10.1007/ s10586-017-1117-8 Liao HJ, Lin CHR, Lin YC, et al (2013) Intrusion detection system: A comprehensive review. Journal of Network and Computer Applications 36(1):16–24. https://doi. org/10.1016/j.jnca.2012.09.004 Lu J, Liu A, Dong F, et al (2019) Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering 31(12):2346–2363. https://doi. org/10.1109/TKDE.2018.2876857 Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems (NeurIPS) Milajerdi SM, Eshete B, Gjomemo R, et al (2019) POIROT: Aligning attack behavior with kernel audit records for cyber threat hunting. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp 1795–1812, https://doi.org/10.1145/3319535.3363217 Miller DR, Harris S, Harper A, et al (2010) Security Information and Event Management (SIEM) Implementation. McGraw-Hill Osborne Media Müller A, Göldi C, Tellenbach B, et al (2009) Event correlation engine. Master’s thesis, ETH Zürich, Communication Systems Group, available at https://pub.tik.ee.ethz. ch/students/2009-FS/MA-2009-01.pdf Niculescu-Mizil A, Caruana R (2005) Predicting good probabilities with supervised learning. In: Proceedings of the 22nd International Conference on Machine Learning (ICML), pp 625–632, https://doi.org/10.1145/1102351.1102430 Nisioti A, Mylonas A, Yoo PD, et al (2018) From intrusion detection to attacker attribution: A comprehensive survey of unsupervised methods. IEEE Communications Surveys & Tutorials 20(4):3369–3388. https://doi.org/10.1109/COMST.2018. 2854724 OWASP Foundation (2021a) OWASP top ten web application security risks. URL https://owasp.org/www-project-top-ten/, accessed: 2022-07-01 OWASP Foundation (2021b) OWASP web security testing guide (WSTG) v4.2. Tech. rep., OWASP Foundation, URL https://owasp.org/ www-project-web-security-testing-guide/
36
Pendlebury F, Pierazzi F, Jordaney R, et al (2019) TESSERACT: Eliminating experimental bias in malware classification across space and time. In: Proceedings of the 28th USENIX Security Symposium, pp 729–746 PortSwigger (2022) Burp suite: Web application security testing. URL https:// portswigger.net/burp, accessed: 2022-07-01 Prokhorenkova L, Gusev G, Vorobev A, et al (2018) CatBoost: Unbiased boosting with categorical features. In: Advances in Neural Information Processing Systems (NeurIPS) Reeves O, Mehlmauer C (2022) Gobuster: Directory/file, DNS and VHost busting tool. URL https://github.com/OJ/gobuster, accessed: 2022-07-01 Ring M, Wunderlich S, Scheuring D, et al (2019) A survey of network-based intrusion detection data sets. Computers & Security 86:147–167. https://doi.org/10.1016/j. cose.2019.06.005 Sarker IH, Abushark YB, Alsolami F, et al (2020) IntruDTree: A machine learning based cyber security intrusion detection model. Symmetry 12(5):754. https://doi. org/10.3390/sym12050754 Settles B (2012) Active Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Morgan & Claypool, https://doi.org/10.2200/ S00429ED1V01Y201207AIM018 Sharafaldin I, Habibi Lashkari A, Ghorbani AA (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. In: Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP), pp 108–116, https://doi.org/10.5220/0006639801080116 Sommer R, Paxson V (2010) Outside the closed world: On using machine learning for network intrusion detection. In: Proceedings of the 2010 IEEE Symposium on Security and Privacy (S&P), pp 305–316, https://doi.org/10.1109/SP.2010.25 sqlmap development team (2022) sqlmap: Automatic SQL injection and database takeover tool. URL https://sqlmap.org, accessed: 2022-07-01 Stallings W, Brown L (2017) Computer Security: Principles and Practice, 4th edn. Pearson Strom BE, Applebaum A, Miller DP, et al (2018) MITRE ATT&CK: Design and philosophy. Tech. Rep. MP180360, The MITRE Corporation Stuttard D, Pinto M (2011) The Web Application Hacker’s Handbook: Finding and Exploiting Security Flaws, 2nd edn. Wiley
37
Tavallaee M, Bagheri E, Lu W, et al (2009) A detailed analysis of the KDD CUP 99 data set. In: Proceedings of the 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), pp 1–6, https://doi.org/10. 1109/CISDA.2009.5356528 Veeramachaneni K, Arnaldo I, Korrapati V, et al (2016) AIˆ2: Training a big data machine to defend. In: 2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), pp 49–54, https://doi.org/10.1109/ BigDataSecurity-HPSC-IDS.2016.79 Vinayakumar R, Alazab M, Soman K, et al (2019) Deep learning approach for intelligent intrusion detection system. IEEE Access 7:41525–41550. https://doi.org/10. 1109/ACCESS.2019.2895334 Wazuh, Inc. (2022) Wazuh: The open source security platform. URL https://wazuh. com, accessed: 2022-08-01 Xiong W, Lagerström R (2019) Threat modeling – a systematic literature review. Computers & Security 84:53–69. https://doi.org/10.1016/j.cose.2019.03.010 Yang L, Ciptadi A, Laziuk I, et al (2021) CADE: Detecting and explaining concept drift samples for security applications. In: Proceedings of the 30th USENIX Security Symposium, pp 3327–3344
38