Risk-Averse Alert Prioritization for IDS Using Subnormal Gaussian Fuzzy Models Murat Moran1
arXiv:2605.27299v1 [cs.CR] 26 May 2026
1
Department of Computer Engineering, Giresun University, Türkiye [email protected]
Abstract Modern intrusion detection systems generate thousands of alerts daily, but alert fatigue severely limits security operations effectiveness due to too many false positives or low-impact events. We address this by proposing a principled framework for alert prioritization based on subnormal Gaussian fuzzy numbers, explicitly modeling three sources of uncertainty: threat severity, detection confidence, and organizational risk attitude. Each alert is represented as a fuzzy number with the core indicating severity, spread indicating uncertainty, and height reflecting detection reliability. We apply ranking indices to prioritize alerts, allowing organizations to tune security posture through a risk-attitude parameter. Experimental validation on CIC-IDS2017 and NSL-KDD demonstrates greater robustness than baselines under detector degradation (0.9963 vs 0.8215 NDCGrel @100), with distinct differentiation in mid-confidence alerts and near-parity with baselines under robust detectors. The framework is theoretically grounded, computationally efficient, provides interpretable reasoning, and remains robust across detector families and miscalibration scenarios.
Keywords: Intrusion Detection System, Alert Prioritization, Risk-Averse Decision-Making, Alert Fatigue, Fuzzy Logic, Cybersecurity Operations, Uncertainty Quantification
1
Introduction
Modern intrusion detection systems (IDS) are essential components of the enterprise cybersecurity infrastructure. However, their effectiveness is severely limited by a critical operational challenge: alert fatigue, i.e., reduced analyst responsiveness caused by excessive security alerts. This issue has been extensively studied in the context of security operations centers (SOCs), where it is recognized as a significant contributor to the failure to detect attacks [1]. SOCs are typically busy with thousands of alerts daily; however, the overwhelming majority are false positives (FPs) or low-severity incidents that do not require immediate attention. Consequently, this high volume of alerts consumes analysts’ ability to prioritize effectively, which in turn results in delayed detection of critical threats, analyst burnout, and substantial operational expenses. The scale of this problem is substantial. Recent surveys indicate that a typical enterprise IDS generates 10,000 or more alerts every day, with more than 50% being FPs [2]. This volume of alerts causes significant alert fatigue among security analysts [3], with 64% of SOC teams reporting being overwhelmed by FPs [4]. Consequently, analysts often ignore alerts, disable rules, or pass them to colleagues [2], potentially missing critical threats. This inefficiency has direct business consequences: increased incident response times, higher breach detection latency, wasted analyst resources, and ultimately reduced security posture. In this context, the ability to accurately rank and prioritize alerts is not merely a convenience, but a necessity for effective cybersecurity. 1
The core problem of alert fatigue is not that IDS fails to find attacks. Instead, it comes from the large number of alerts generated by current systems, along with a lack of effective prioritization. This prioritization issue is essentially a problem of decision making under uncertainty. Each alert has several related sources of uncertainty that current systems do not handle well. These include: Severity Uncertainty: The risk associated with different attacks varies. A vulnerability within a critical system presents a greater risk than one in a non-critical system. Nevertheless, the assessment of severity is itself uncertain due to several factors: 1. The actual impact depends on the organization’s specific infrastructure and business context 2. Standard severity metrics, such as the Common Vulnerability Scoring System (CVSS), are generic and may not reflect organizational risk 3. Novel attacks may not have established severity assessments Detection Confidence Uncertainty: Not all alerts are equally reliable. The confidence that an alert represents a true attack depends on the detection method (feature set, training data, and model choice), the attack class, the specific network context, and whether the attack is novel/zero-day (where validation evidence is limited). Classification Uncertainty: When an IDS flags an alert, it is not always clear what kind of attack is actually taking place: it could be a SQL injection, a buffer overflow, or something entirely different. This uncertainty then affects how we assess the severity of the threat, which in turn impacts how we prioritize our responses.
1.1
Limitations of Existing Approaches
Current alert ranking methods treat these uncertainties implicitly or ignore them entirely. They produce single-point estimates (e.g., a severity score or confidence score) without quantifying the uncertainty around those estimates. This leads to suboptimal prioritization: high-severity but low-confidence alerts (potential zero-days) may be ranked below low-severity, high-confidence alerts, or vise-versa, depending on the ad-hoc weighting scheme used. A principled approach to alert prioritization must explicitly model and reason about these multiple sources of uncertainty. There are different ways to prioritize alerts, but each has its own set of problems. Severitybased ranking uses common metrics such as CVSS scores. But it does not take into account how sure the detection is, which means that analysts might waste time on alerts that aren’t very reliable, even if they are very serious. Conversely, Confidence-based ranking emphasizes alerts generated by reliable detection techniques. However, this method overlooks the severity, which could result in the omission of crucial attacks that are more challenging to identify with precision. An alternative strategy involves Ad-hoc weighted combinations. These methods attempt to reconcile severity and confidence through the application of a straightforward weighted sum or product. However, the weights are arbitrarily chosen, without any theoretical basis, and do not reflect the organization’s specific risk preferences. Lastly, Machine learning approaches learn prioritization from historical data, but are typically black-box, lack interpretability, and require expensive labeled training data [5]. As a result, none of these approaches explicitly model the uncertainty inherent in severity and confidence assessments. They treat these quantities as point estimates rather than uncertain quantities with distributions and confidence intervals. As a result, they cannot distinguish between high-confidence estimates and low-confidence estimates, nor can they adapt their decision-making based on organizational risk tolerance.
2
1.2
Fuzzy Logic and Risk-Averse Decision-Making as a Solution
We present a natural, theoretically sound framework for managing multiple sources of uncertainty in alert prioritization, employing fuzzy logic. Unlike probabilistic methods, which require exact probability distributions, fuzzy logic is capable of processing incomplete data and expert evaluations. Subnormal fuzzy sets, which have a maximum membership degree less than 1, are particularly useful for this problem. They naturally represent the idea of partial reliability. For example, a high-severity alert from a detection method that is not very reliable can be modeled as a subnormal fuzzy set. This set would have a high core value (indicating severity) but a low height (indicating reliability). In addition, ranking indices offer a structured method to reconcile severity, confidence, and an organization’s risk profile. These indices, rather than yielding a singular "best" ranking, enable an organization to adjust its alert prioritization strategy according to its risk appetite. Consequently, a more conservative organization might prioritize alerts with high confidence, whereas a risk-aggressive organization could choose to prioritize alerts with high severity.
1.3
Contributions and Organization
This paper makes four main contributions: 1. Novel Application: We propose the first application of subnormal Gaussian fuzzy numbers to intrusion detection alert prioritization. 2. Framework: We present an alert ranking framework, based on a clear understanding of risk and a parameterized ranking index. It explicitly considers the severity of alerts, the confidence in their detection, and the organization’s risk tolerance. 3. Empirical Validation: We validate the framework using real-world data from the CICIDS2017 dataset. Our results show that we see significant improvements in performance when the alert data is noisy or uncertain. 4. Practical Insights: This research offers a detailed analysis of how key factors, such as detection confidence and risk attitude, affect alert prioritization. As a result, this research provides practical guidance for organizations using this system. The remainder of this paper is organized as follows: Section 2 reviews related work on alert prioritization and fuzzy logic in cybersecurity; Section 3 presents mathematical preliminaries, problem formulation, and the proposed framework; Section 4 details the experimental setup; Section 5 presents results and analysis; and Section 6 concludes with directions for future work.
2
Related Work
IDSs have evolved significantly over the past two decades. Early IDS such as Snort and Suricata employed signature-based detection, matching network traffic with a database of known attack patterns [6]. In recent decades, machine learning approaches have been applied to IDSs, using techniques such as Random Forests, neural networks, and Support Vector Machines (SVM) to detect anomalous behavior [5, 7, 8]. More recently, deep neural networks [9], recurrent neural networks for sequential attack detection [10]. These advances have improved detection accuracy, but have also increased the problem of alert volume — machine learning-based IDS often generate more alerts than signature-based systems because they are more sensitive to deviations from normal behavior. The alert fatigue problem has been extensively documented in the security operations literature. Julisch [11] demonstrated that alert correlation can significantly reduce alert volume, 3
but the fundamental problem of prioritization remains. In recent years, Alahmadi et al. [3] conducted a qualitative study that examined the perspectives of SOC analysts on security alarms, finding that alert validation is a tedious task that causes burnout. They identified five critical properties–Reliable, Explainable, Analytical, Contextual, Transferable–that are missing from current security tools. Jalalvand et al. [2] provided a comprehensive survey of alert prioritization methods, noting that organizations receive more than 10, 000 alerts daily with 50% FPs. In practice, alert prioritization is often performed manually by security analysts using domain knowledge and experience. Despite the operational importance of alert prioritization, this comprehensive systematic survey also confirms that there is limited research on this problem. Recent work in machine learning-based security has emphasized the importance of uncertainty quantification. Bayesian deep learning approaches have been applied to IDS to provide confidence estimates for predictions [12]. Malhotra et al. [13] applied uncertainty quantification to anomaly detection in time series. Guo et al. [14] studied calibration of neural network predictions for security applications. Deep learning methods often treat uncertainty through probabilistic outputs or Bayesian approximations. A notable limitation of these probabilistic frameworks is that they require precise probability distributions, which may not be available in practice. Traditional detection methods often struggle to accurately identify new or previously unknown attacks. A significant limitation of IDSs that use machine learning, particularly deep learning, is their lack of interpretability. Security analysts, who need to trust the system, often need to understand the reasons behind an alert and why it was given a specific priority. Ribeiro et al. [15] developed LIME, a method to explain how machine learning models make predictions. Montavon et al. [16] surveyed interpretability methods for deep learning. However, fuzzy logic offers inherent interpretability. That is, membership functions and ranking indices have clear linguistic interpretations, and analysts can understand the reasoning behind alert prioritization [17]. Our fuzzy-based alert prioritization framework provides interpretable reasoning about alert priorities, complementing the detection capabilities of ML-based IDS. CVSS is the standard metric for assessing vulnerability severity. Mell et al. [18] introduced CVSS v2, which has become widely adopted. The CVSS v3 specification [19] improved on v2 with additional metrics. Despite its ubiquity, CVSS provides only a direct estimate of severity and does not account for organizational context or uncertainty. Allodi and Massacci [20] studied the relationship between CVSS scores and the actual availability of exploits. Jacobs et al. [21] analyzed the limitations of CVSS for vulnerability prioritization. Our approach extends CVSS by incorporating the organizational context through contextual factors and modeling uncertainty through the spread parameter (σ) of the fuzzy number. Julisch [11] pioneered alert correlation using clustering techniques. Alert correlation groups related alerts into clusters to reduce alert volume, while our work focuses on ranking individual alerts by priority. Although complementary, correlation alone does not solve the prioritization problem, even after clustering, analysts must still decide which alerts or clusters to investigate first. Our framework provides fine-grained per-alert prioritization that can be applied after correlation. Fuzzy logic has been used in various cybersecurity areas, including finding unusual activities, assessing threats, and detecting intrusions. Fuzzy logic can work with partial information and expert assessments, provides interpretable reasoning, and naturally represents the concept of partial membership and partial reliability [17]. Tsang et al. [22] proposed a genetic-fuzzy rule mining system for intrusion detection. More recently, Kar and Mishra [23] developed a fuzzy-based approach to detect HTTP-based attacks. Meng et al. [24] applied fuzzy logic to alert prioritization in network security, but their approach did not explicitly model detection confidence or use subnormal fuzzy sets. Existing applications of fuzzy logic to IDS typically do not explicitly address alert prioritization, nor do they model detection confidence or organizational risk attitude. Our framework based on subnormal Gaussian fuzzy numbers (SGFN) 4
Figure 1: The framework of the proposed intrusion detection alert prioritization system. provides a more interpretable model of uncertainty. This allows security analysts to more easily understand and even tune the system’s risk-attitude based on the three intuitive sources of uncertainty: severity, detection confidence, and classification ambiguity. Additionally, our framework naturally handles novel attacks by assigning low detection confidence, which can be adjusted based on organizational risk tolerance. It is complementary to the detection-focused approaches mentioned above. With this combination, organizations can achieve both high detection accuracy and effective alert management. To this end, we use a machine learning baseline IDS and focus on the main problem of ranking and prioritizing its alerts.
3
Methodology
3.1
Overview of the Proposed Framework
We propose a comprehensive framework for IDS alert prioritization based on SGFNs and ranking indices. The framework has four steps: (1) represent alerts as fuzzy numbers, (2) extract and calibrate parameters, (3) calculate risk-averse rankings, and (4) rank alerts. Figure 1 illustrates the overall architecture of the proposed system: Raw alerts from a baseline IDS are converted to SGFN with three parameters: core value c (threat severity), spread value σ (uncertainty in impact) and height value h (detection confidence). The fuzzy ranking engine applies the ranking formula Rbenef it = c+κσ log(h) to compute priority scores. The risk-attitude parameter κ, which we sweep over 0–2 (with the standard risk parameter κ = 1 as the default in our main results), allows organizations to customize their alert prioritization based on their risk tolerance. After this, the alerts are sorted by their priority scores and presented to SOC analysts in a prioritized queue, which helps them investigate threats more effectively. The primary advancement of our methodology lies in its explicit incorporation of three distinct sources of uncertainty within the alert prioritization process: uncertainty regarding 5
threat severity, uncertainty regarding detection confidence, and the organization’s risk attitude. In contrast to conventional methods, our framework offers a structured approach for evaluating alerts under uncertainty, enabling organizations to adjust their security posture according to their specific risk tolerance levels.
3.2
Mathematical Preliminaries and Definitions
3.2.1
Subnormal Gaussian Fuzzy Numbers (SGFNs)
Given that p, q, r ∈ R, where p < q < r and h ∈ (0, 1], an SGFN à on R is a fuzzy subset characterized by a membership function µÃ : R → [0, 1] satisfying the following properties [25, 26]: 1. µÃ is continuous 2. µÃ (x) = 0 for all x ∈ / [p, r] (compact support) 3. µÃ is strictly increasing on [p, q] and strictly decreasing on [q, r] 4. µÃ (q) = h where h ∈ (0, 1] (height parameter) An SGFN à = ⟨(cà , σà ); hà ⟩ has a Gaussian membership function defined as:
µÃ (x) = hà exp −
1 2
x − cà σÃ
!2
(1)
where: • cà ∈ R is the center (core), representing the most likely value • σà > 0 is the spread (standard deviation), representing uncertainty or dispersion • hà ∈ (0, 1] is the height, representing the maximum membership degree (reliability) The Gaussian membership function has several desirable properties: it is smooth, has a compact parametric representation, and supports closed-form arithmetic operations. 3.2.2
α-Cut Representation
The α-level set (or α-cut) of an SGFN à is the set of all elements with membership degree at least α: Ãα = {x ∈ R | µÃ (x) ≥ α, α ∈ [0, hà ]} = [ÃL (α), ÃR (α)]
(2)
For an SGFN with Gaussian membership function, the left and right endpoints of the α-cut are: q ÃL (α) = cà − σà −2 ln(α/hà ) (3) q
ÃR (α) = cà + σà −2 ln(α/hà )
(4)
The α-cut representation is useful for understanding the range of possible values at different levels of confidence.
6
3.2.3
Ranking Index for Fuzzy Numbers
For a benefit-type fuzzy number (where higher values are preferable), we use the ranking index proposed in our prior work [27]. Rbenefit (Ã; κ) =
c
à |{z}
base severity
+ κσà log(hà ) |
{z
(5)
}
risk adjustment
where κ ≥ 0 is the risk-attitude parameter. The first term is the base severity (core value). The second term is a risk adjustment that penalizes alerts with high uncertainty (σà ) or with low confidence (hà ) since the value log(hà ) is negative (assuming base-10 logarithm). This penalty is also controlled by the risk-attitude parameter κ.
3.3
Alert Representation as SGFNs
The fundamental advancement of our methodology is the representation of each IDS warning as an SGFN. This representation captures three key aspects of alert assessment: severity, uncertainty, and detection confidence. 3.3.1
Core Value ci (Threat Severity)
The core value ci of the alert Ai represents the threat severity score. We compute this using the CVSS score of the corresponding attack type and organizational context (contextual factor cfi ): ci = CVSS(attack_typei ) × cfi (6) where: • CVSS(attack_typei ) ∈ [0, 10] is the CVSS base score for the detected attack type, obtained from the National Vulnerability Database (NVD) or threat intelligence sources • cfi ∈ [0, 1] is an organizational context factor that adjusts severity, based on whether the target system is critical CVSS provides a numerical score, based on factors such as attack vector, required privileges, user interaction, and potential impact. Security teams prioritize vulnerabilities according to CVSS scores for attack types. For example, a remote SQL injection vulnerability that requires no authentication and allows full database compromise might receive a CVSS score of 9.8 (Critical) due to its significant impact on confidentiality, integrity, and availability (CIA). A stored cross-site scripting (XSS) vulnerability that necessitates user interaction but facilitates session hijacking could be assessed at approximately 7.2, indicating a high severity. Conversely, a reflected XSS vulnerability, characterized by a restricted impact and the requirement of user interaction, might receive a medium rating of 5.4. Furthermore, a local denial-of-service vulnerability, which exclusively compromises availability, could be assigned a low score of 3.1. The contextual factor cfi is the environment-specific conditions that influence how severe or risky a vulnerability is for a particular organization, beyond its generic technical severity. Thus, it means that the same vulnerability can have different risk levels for different organizations. Organizations generally use tools to calculate these scores with respect to their modified impact metrics on CIA and to weigh them according to their security requirements. Throughout the paper, we consider the values of cfi as the following:
cfi =
1.0 0.8 0.5
0.2
if target is critical infrastructure if target is important business system if target is non-critical system if target is isolated test system 7
(7)
This enables the framework to adapt to organizational priorities. For example, an attack on a critical system is weighted more heavily than the same attack on a non-critical system. In our experiments, we assign a deterministic per-alert contextual factor cfi ∈ [0.2, 1.0] using a hash of the alert id and attack type, then map it to the nearest categorization. This increases contextual variability without introducing randomness. 3.3.2
Spread Value σi (Uncertainty in Severity)
The spread value σi represents uncertainty in severity assessment. We compute this as a percentage of the core value by multiplying it by the uncertainty factor ufi . Its value depends on the attack types and reflects our confidence in the severity assessment. Well-known attacks have well-understood impacts, and thus low uncertainty. Novel attacks have unknown impacts and thus high uncertainty. σi = ci × ufi (attack_typei ) (8) where ufi (attack_typei ) ∈ [0, 0.5] depends on the attack class i and σi is the standard deviation derived by multiplying the uncertainty factor and severity value. These values are assigned based on how well-known the attack is. For instance, values may be 0.1 for well-known attacks (e.g. SQL injection, buffer overflow, brute force, port scan), 0.2 for moderately known attacks (e.g., specific DoS and DDoS variants, malware), 0.35 for emerging and poorly known threats (e.g., new malware variants, anomalous traffic), and 0.5 for novel or zero-day attacks. Although this approach to calculate σi requires defining the value uf for each attack type, it is proportional to severity: higher severity means higher uncertainty. 1
ufi (attack_typei ) =
0.1 0.2
if it is well-known if it is moderately known if it is emerging if it is novel/zero-day
0.35
0.5
3.3.3
(9)
Height Value hi (Detection Confidence)
The height value hi ∈ [0, 1.0] encodes the detector reliability for this alert (the height of the fuzzy number in the SGFN representation), not the raw confidence score pi from the baseline IDS. We first compute class-level hclass from validation performance (Equation 10). the height parameter reflects the actual historical reliability of detecting that attack class. Hence, the type of detection method that generated the alert is substantial. For instance, if the detection method is reliable with few FPs (signature-based methods such as Snort rules) then the height value should be high; If the detection method is moderately reliable with many FPs (heuristic based, machine learning, and anomaly detection) then the height value should be around median; Lastly, if the detection method is unreliable with too many FPs (novel behavior) then the height value should be low. However, before determining instance-level detection confidence hi , we first compute the class-level height value hclass for each attack type depending on the historical performance of the detection method during IDS training on a validation dataset as in Equation 10. hclass = clip (0.5 + α(F1i − 0.5), hmin , hmax ) (10) We clamp 0.5 + α(F1i − 0.5) to [hmin , hmax ] to avoid saturation and use α = 0.9, hmin = 0.05, hmax = 0.95 in our experiments. α controls shrinkage strength (1.0 means no shrinkage; smaller values apply stronger smoothing) and reduces variance in small or imbalanced classes. Appendix A.6 reports a sensitivity analysis for α, hmin , hmax , and a global uncertainty scaling factor, showing that the ranking results are stable within reasonable ranges. 1
For each attack type, corporations need to validate their calculated uncertainty scores to see whether it matches with their experience and adjust accordingly if necessary as in Definition 9.
8
To determine hi we retain instance-level variability by capping the class-level height with the per-alert model confidence pi as in Equation 11. Using the min implements a conservative cap: an individual alert cannot be more reliable than the class has proven to be, and a low peralert confidence should down-weight even a strong class. It is conservative in a principled way: the system refuses to trust an alert beyond what historical performance and current evidence jointly justify. hi = min(hclass , pi ) (11) where pi = P (attack | xi ) denotes the IDS’s raw probability for alert i, showing how certain this alert is an attack. This may be different for each alert of the same class and varies unpredictably. Precisioni is the precision of the baseline IDS in this attack class (fraction of alerts that are true positives). Although it requires manual verification and historical data, this is the most accurate method and reflects actual IDS performance. Here, i indexes the attack class, T Pi is the number of true positives, F Pi is false positives, and F Ni is false negatives. Precisioni =
T Pi T Pi + F Pi
(12)
Recalli also accounts for false negatives of the baseline IDS. Recalli =
T Pi T Pi + F Ni
(13)
F1i is the F1-score of the baseline IDS in this attack class (harmonic mean of precision and recall). This is particularly important for detection quality because precision alone ignores missed detections, and hence F1 balances FPs and FNs: F1i = 2 ×
Precisioni × Recalli Precisioni + Recalli
(14)
We include precision and recall for diagnostics, but we use F1 to derive the class height using Equation 10. For attack classes not seen during validation (novel attacks), we use a conservative estimate hclass = 0.5 and then apply the same capping as pi . This reflects the fact that we have no empirical evidence of the reliability of the detection method on novel attacks. The key insight is that the height parameter captures the reliability of the detection method, not the raw confidence score from the IDS. This is crucial because 1. a high-confidence alert from an unreliable detection method should be de-prioritized 2. a low-confidence alert from a reliable detection method should be prioritized 3. the height parameter is attack-class specific, allowing different reliability for different attack classes To illustrate the representation of alerts as SGFNs, assume a raw alert from an MLbased anomaly detector with attack type SQL Injection, CVSS score 7.5, and target customer database. The calculations for the fuzzy number ⟨(c, σ); h)⟩ are: • Contextual Factor: 0.8 (important target), hence c = 7.5 × 0.8 = 6.00 • Uncertainty Factor: 0.1 (well-known attack), hence σ = 6.00 × 0.15 = 0.90 • Height Value: Let us assume historical data as: 45 TP, 35 FP, 15 FN, so Precision = 45/(45+35) = 0.56 and Recall = 45/(45+15) = 0.75, F1 = 2×(0.56×0.75)/(0.56+0.75) = 0.64. With α = 0.9, hclass = 0.5 + 0.9(0.64 − 0.5) = 0.626 (clipped to [0.05, 0.95]); the height of the instance is hi = min(hclass , pi ). 9
Algorithm 1 Context-Aware Alert Representation and Ranking Require: Alert set A = {A1 , A2 , . . . , AN }, uncertainty factors, CVSS scores, target criticality to derive the contextual factor, detection metrics, height smoothing parameters (α, hmin , hmax ), parameter κ Ensure: Ranked list of alerts 1: Step 1: Alert Representation 2: for each alert Ai ∈ A do 3: Determine attack type attack_typei 4: Retrieve CVSS base score CVSSi ← CVSS(attack_typei ) 5: Compute contextual factor cfi ← cfi (Ai ) 6: Compute core severity value ci ← CVSSi × cfi 7: Compute spread value σi ← ci × ufi (attack_typei ) 8: if attack type is known then 9: Set class confidence hclass ← clip(0.5 + α(F1(attack_typei ) − 0.5), hmin , hmax ) 10: Set detection confidence hi ← min(hclass , pi ) 11: else 12: Set detection confidence hi ← min(0.5, pi ) 13: end if 14: Construct SGFN alert representation Ãi ← ⟨(ci , σi ); hi ⟩ 15: end for 16: Step 2: Ranking Score Computation 17: for each alert representation Ãi do 18: Compute ranking score Ririsk-averse ← ci + κσi log(hi ) 19: end for 20: Step 3: Alert Sorting 21: Sort alerts in descending order by ranking score Ririsk-averse 22: Step 4: Alert Presentation 23: for each alert in sorted order do
Present attack type, target system, ci , σi , hi , and Ririsk-averse 25: end for 26: return Ranked alert list 24:
3.4
Alert Prioritization Algorithm
Given a set of alerts A = {A1 , A2 , . . . , An } and a risk-attitude parameter κ, we first convert each raw alert into an SGFN. Second, we calculate the priority score using the ranking formula in Equation 5. Here, the ranking score is a single numerical value that combines severity, uncertainty, and detection confidence into a unified prioritization metric. Third, we sort the alerts by priority score in descending order so that the alerts with higher ranking scores have higher priority. Finally, we present ranked alerts to analysts with interpretable information. More details of each step of the alert prioritization algorithm are given in Algorithm 1. About the computational complexity of the alert prioritization algorithm: Given that n is the number of alerts, the complexity of the representation of alerts and ranking score computation is linear O(n). Moreover, sorting in terms of ranking scores is O(n log n). The overall complexity is O(n log n), which is efficient even for large numbers of alerts.
3.5
Interpretation of the Ranking Score
The ranking score computed for each alert Ribenefit (Ãi ; κ) = ci +κσi log(hi ), we call it Ririsk-averse , has a clear interpretation: 10
Base Severity Term (ci ): This is the main driver of alert priority. Higher severity alerts are prioritized. Risk-Attitude Parameter (κ): This parameter allows organizations to tune their security posture. Table 1 describes what different κ values mean. Table 1: Risk Aversion Profiles based on κ values Risk Profile Description
Value of κ κ = 0.0 κ = 0.5 κ = 1.0 κ > 1.0
Pure Severitybased Moderate Risk Aversion Standard Risk Aversion High Risk Aversion
Aggressive; focuses on high severity regardless of confidence. Balanced approach between severity and confidence. Default; balances severity and confidence equally. Conservative; prioritizes high-confidence alerts over severity.
Risk Adjustment Term (κσi log(hi )): This term adjusts the priority based on the riskattitude parameter, uncertainty, and confidence. Since hi ∈ (0, 1], we have log(hi ) ≤ 0, hence the lower confidence value hi means the bigger negative number, which penalizes more. As κσi log(hi ) ≤ 0, meaning that the adjustment term is negative; higher risk-aversion, higher uncertainty, and lower confidence reduce the priority of alerts with higher penalties.
3.6 3.6.1
Handling Edge Cases and Calibration Novel or Zero-Day Attacks
For attacks not seen during validation, we cannot compute the detection confidence from historical data. We handle this by assigning a conservative class height: hclass = 0.5 and applying hi = min(hclass , pi ) 3.6.2
Missing or Incomplete Information
If some information is missing (e.g., CVSS score not available), the default values shown in Table 2 can be used: Table 2: Default values for missing vulnerability metrics Missing Parameter CVSS Score (CVSSi ) Contextual Factor (cfi ) Detection Confidence (hi ) IDS Attack Probability (pi )
3.6.3
Default Value 5.0 0.5 0.5 0.5
Interpretation Medium severity Moderate importance Moderate reliability Default per-alert attack probability
Calibrating the Risk-Attitude Parameter (κ)
The risk-attitude parameter κ should be calibrated based on organizational risk tolerance. Thus, an organization would start with κ = 1 (default, balanced) and analyze the top-ranked alerts over a period of time. If too many FPs are in the top-ranked alerts, the analyst can increase κ (more conservative) and if critical attacks are being missed, s/he can decrease it (more aggressive). This can be repeated until the ranking aligns with expectations. 11
3.6.4
Calibrating Detection Confidence (hi )
The detection confidence should be computed from a validation data set. Thus, we first train the baseline IDS in the training set, then evaluate in the validation set (15-20% of the data). Next, we compute precision, recall, and F1 for each attack class and derive hclass from the smoothed F1 (Equation 10), then cap with pi using Equation 11. For novel attacks, we start from hclass = 0.5 (smoothed) and update as more data become available.
3.7
Integration with Existing IDS
Our framework is designed to work with any baseline IDS (signature-based, ML-based, or hybrid). During the integration process, the baseline IDS generates alerts with the attack type and confidence score. Our framework reads these alerts and extracts the attack type. For each alert, we look up the CVSS score, contextual factor, uncertainty factor, and detection confidence, compute the ranking score, and present ranked alerts to the analyst. The framework can be deployed as a post-processing module on top of existing IDS. This modular design allows organizations to adopt our framework without replacing their existing IDS infrastructure.
3.8
Comparison with Baseline Ranking Approaches
Our framework combines severity, uncertainty, and confidence with our underlying ranking as in Equation 5. This approach is theoretically sound, allows for adjustments to risk preferences, and explicitly considers uncertainty. To better understand our method, we compare it with three standard ranking techniques: 1. Baseline 1: Severity-Only Ranking: Ranks alerts purely by CVSS score: Riseverity = ci . This ignores detection confidence and leads to wasted analyst time investigating lowconfidence, high-severity alerts. 2. Baseline 2: Confidence-Only Ranking: Uses only confidence for ranking, sorts alerts based solely on the model’s confidence for each alert: Riconfidence = pi . This method does not consider severity, which could lead to missing important attacks that are harder to detect accurately. 3. Baseline 3: Weighted Sum Ranking: Combines severity and per-alert confidence with ad-hoc weights: Riweighted = 0.5norm(ci ) + 0.5norm(pi ). This does not account for the organizational risk attitude or uncertainty.
4
Experimental Setup
4.1
Dataset
We evaluate our framework using the CIC-IDS2017 dataset [28]. The dataset is a publicly available, modern, comprehensive intrusion detection dataset created by the Canadian Institute for Cybersecurity. It has been widely used in the literature, allowing us to compare our framework with other related works. It contains real-world network traffic captured over five days in a controlled environment. The dataset is ideal for our evaluation because it contains diverse, realistic attack types (both common attacks and emerging threats), and provides fine-grained flow-level labels. The data set consists of 2,830,743 total network flows, collected in 5 continuous day network traffic, 80.3% of which are benign (2,273,643) and 19.7% of which are attack flows (557,100). It also covers 78 network flow features (source/destination IP, ports, protocol, packet statistics, etc.) and 11 attack types (mapped to 8 classes); Table 3 shows the condensed categories. For data preparation and splitting, we follow standard practice for IDS evaluation: 12
Table 3: Classification of Common Cyber Attack Types Category
Attack Type
Description
Credential Guessing / Password Attacks
Brute Force (FTP-Patator, SSH-Patator)
Repeated, systematic attempts to guess login credentials for network services such as FTP and SSH.
System Application Attacks
Web Attacks (Brute Force, XSS, SQL Injection)
Exploitation of web application vulnerabilities, including brute-force login attempts and injection attacks.
Vulnerability Exploitation
Heartbleed
Exploitation of a cryptographic vulnerability in OpenSSL that allows attackers to read sensitive memory contents.
Denial of Service
DoS (Hulk, GoldenEye, Slowloris, Slowhttptest)
Resource-exhaustion attacks that degrade or deny service availability.
Distributed Denial of Service
DDoS
Coordinated attacks that overwhelm a target with excessive traffic from multiple sources.
Reconnaissance
Port Scan
Probing network ports to discover open services and potential weaknesses.
Malware / Botnets
Bot
Botnet-controlled traffic used for spam, scanning, or coordinated attacks.
Lateral Movement / Intrusion
Infiltration
Unauthorized internal access or data exfiltration after initial compromise.
• Feature Normalization: We normalize all 78 features to the range [0, 1] using min-max scaling to ensure that no single feature dominates due to scale differences. • Train/Validation/Test Split: Sort day files by weekday. If the day-based train split (Mon/Tue) has at least 5% attacks, use Mon/Tue for training, Wed for validation, Thu/Fri for test. • Attack Type Mapping: Map the 11 attack types to 8 attack classes by grouping variants that represent the same attack family to consolidate many raw labels into a smaller and consistent set of attack classes • Imbalance Handling: Keep the natural imbalance to avoid the model learning "everything is benign" when attacks are rare (below 5%) unless the fallback is triggered to a stratified split (50/20/30) to preserve learnability.
4.2
Baseline IDS
To evaluate our alert prioritization framework, we use a machine learning-based baseline IDS Logistic Regression (LR) to generate alerts. The baseline IDS is trained to detect attacks and is used to produce confidence scores, which are later used to rank alerts in our prioritization framework. To this end, we employ Logistic Regression (LR) as the baseline IDS because it provides well-calibrated probabilities suitable for ranking analysis, offers interpretability through linear coefficients, and efficiently handles class imbalance. Importantly, when detectors are nearperfect (such as Random Forest), different ranking methods converge, making comparisons 13
uninformative. A moderately-performing detector, such as LR, better demonstrates the value of ranking under uncertainty. The LR hyper-parameters are solver saga, C = 1.0, maximum iterations = 1000 and class weights set to balanced. We apply sigmoid probability calibration on the validation set and use the calibrated class probabilities as per-alert confidence pi . We also evaluate auxiliary detectors, Random Forest and Isolation Forest, in Appendix A.5 to check detector sensitivity. For each attack class in the validation data set, we compute detection confidence metrics (precision, recall, and F1) and derive the class-level height hclass using the smoothed F1 formulation in Equation 10. During ranking, we use hi = min(hclass , pi ) to cap the class reliability by the per-alert confidence. To stress the detector without changing the ranking logic, we also train a reduced-feature LR using only TCP-flag-related attributes (flags-only). This produces a deliberately noisier alert stream while keeping the ranking pipeline identical, enabling clearer comparison of ranking methods under realistic uncertainty.
4.3
Performance Metrics
To measure the quality of the ranking while accounting for the position of items in the ranked list, we use Normalized Discounted Cumulative Gain (NDCG) [29]. Unlike binary metrics (true vs. false), NDCG can account for severity levels. It rewards systems more for placing "Critical" alerts at the top than "Medium" ones. NDCG ranges from 0 to 1, where 1 represents a perfect ranking (the higher, the better). It is particularly useful because it accounts for ranking position: finding a TP at position 1 is much better than finding it at position 100. NDCGrel @k is a severity-adjusted variant. Using NDCGrel @k, we directly measure what matters operationally: whether our ranking method successfully brings the most important, true attacks to the top of the alert queue. DCG@k =
k X 2reli − 1 i=1
log2 (i + 1)
(15)
DCG@k (16) IDCG@k where reli is the relevance of the item at position i (1 for TP 0 for FP), and IDCG@k is the DCG of the hypothetical perfect or ideal ranking (all TPs first). To emphasize severity while accounting for uncertainty, we adjust the original definition of relevance by severity: for TPs ci (1 − ufi ) and 0 for benign alerts. We denote the resulting metric by NDCGrel @k and present multiple cutoffs. NDCGrel @10 is the primary operational metric, reflecting the author’s concept of an "impatient user" who only looks at the top results [29]. Reasons to choose this metric: (1) most SOC analysts review 5 to 15 alerts per session, making the top 10 list the operational focus; (2) this cutoff best reveals where ranking decisions matter most—at the boundary between high-severity and borderline alerts; (3) larger k values compress differences because the predicted-alert queue is attack-dominant, so the top 100 are nearly always true attacks. Moreover, we make use of NDCGrel @50 and @100 to assess robustness beyond the top list, and NDCGrel @500 for bootstrap significance testing. This multi-faceted approach allows for an evaluation of both the immediate usefulness of the ranking methods in practice and their overall stability. We also evaluate the performance of the rankings within the confidence bands of the IDS probability pi . We partition the alerts into three bins (0.3–0.5, 0.5–0.7, 0.7–1.0) and compute NDCGrel @k on the band-restricted rankings. This highlights the differences in ranking methods when the detector’s certainty is low. In addition, we evaluate the sensitivity of the height-smoothing and uncertainty parameters by sweeping α, hmin , hmax , and a global scaling of ufi in the full-feature LR setting. ApNDCG@k =
14
pendix A.6 shows NDCGrel @10pred and NDCGrel @100pred across these ranges, presenting only minor variation, indicating that the ranking behavior is stable within reasonable limits.
4.4
Baseline Ranking Approaches
We compare our ranking method with three baseline ranking approaches that represent different levels of sophistication in alert prioritization. 4.4.1
Baseline 1: Severity-Only Ranking
This method ranks alerts solely on the basis of CVSS-based severity ci ∈ [0, 10]. It is simple and transparent, but ignores detection confidence and can elevate low-confidence, high-severity alerts. Riseverity = ci (17) 4.4.2
Baseline 2: Confidence-Only Ranking
This method ranks alerts purely by the raw confidence score pi ∈ [0, 1] from the IDS. It prioritizes alerts the detector is most confident about, which can reduce FPs, but ignores severity and can miss critical attacks that are harder to detect. For unsupervised detectors (Isolation Forest in our experiments), anomaly scores are not calibrated probabilities, so Confidence-Only is a diagnostic baseline rather than a recommended method. Riconfidence = pi 4.4.3
(18)
Baseline 3: Weighted Sum Ranking
This method combines severity and detector confidence with fixed weights (equal weights by default). It provides a middle ground between Baseline 1 and 2, but the weights are ad hoc and do not reflect the organizational risk attitude. To ensure a fair comparison between severity and confidence components, we apply scale normalization as in Equation 19. This normalization ensures that both severity and confidence contribute meaningfully to the ranking score, preventing one component from dominating due to scale mismatch (CVSS severity typically ranges [5, 10] with variance ∼1.5, while detector confidence ranges [0.5, 0.95] with variance ∼0.03 for predicted alerts). Riweighted = 0.5norm(ci ) + 0.5norm(pi )
(19)
ci −min(ci ) where norm(ci ) = max(c normalizes the CVSS severity range and norm(pi ) is also i )−min(ci ) calculated in the same way, normalizing pi values.
4.4.4
Proposed Method: Risk-Averse Ranking
In our method, we rank alerts using the ranking index as in 5. It explicitly models uncertainty and allows organizations to tune their risk attitude. Similarly, with this method, higher score equals to higher priority. Thus, it favors alerts with high severity and high confidence, and down-weights alerts with high uncertainty or low confidence, in a controllable way. Although the method requires historical detection data to compute height values and requires parameter calibration, it explicitly models uncertainty in severity assessment, allowing tuning of risk attitude through parameter κ, and provides interpretable reasoning about alert priorities.
15
5
Results and Analysis
5.1
Baseline IDS Performance
We evaluate the baseline IDS using standard classification metrics in the CIC-IDS2017 validation split to show the alert generator is competent. Additionally, we include a stressed LR trained in a reduced feature subset (flags-only) to create a more noisy alert stream for the analysis, presented in 5.3. Table 4 shows the IDS performance for the two different environments. Table 4: Baseline IDS validation performance on CIC-IDS2017 (full vs. flags-only LR) Features Full (all features) Flags-only (stress)
5.2
Accuracy
Precision
Recall
F1-score
0.9268 0.8089
0.8382 0.5822
0.7789 0.1055
0.8075 0.1786
Ranking Performance
We first evaluate the ranking performance on the full-feature LR detector, presented in Table 5. This scenario establishes a baseline understanding of method performance under favorable conditions. Our evaluation focuses on predicted-attack alerts (true attacks alerted by the IDS) to isolate the analyst’s review queue and to avoid benign dominance in the metrics. Table 5: Predicted-alert ranking performance on CIC-IDS2017 (full-feature LR) Method
NDCGrel @10pred NDCGrel @50pred NDCGrel @100pred NDCGrel @500pred
Severity-Only Confidence-Only Weighted Sum Risk-Averse (κ = 1)
0.9534 0.5858 0.9534 1.0000
0.9836 0.5858 0.9836 1.0000
0.9899 0.5858 0.9899 1.0000
0.9970 0.5858 0.9970 0.9974
With the full-feature detector, differences are intentionally small: Risk-Averse reaches NDCGrel @10 = 1.0000 (+0.0466 over Severity-Only and Weighted Sum, +0.4142 over Confidence-Only), illustrating near-parity under strong IDS. A key finding in this scenario is that the Risk-Averse method with κ = 1 achieves a perfect NDCGrel @10 score of 1.0000. This perfect score is not a trivial result; it signifies that the method successfully ranked the top-10 most critical and true attacks in the exact order of their severity, without errors. This performance is attributed to the use of high-quality confidence scores (hi ) provided by the full-feature detector. The risk-attitude parameter (κ = 1) provides a balanced default boost to high-severity alerts that also have high confidence, effectively breaking ties and correcting minor mis-orderings that a purely severity-based ranking might produce. For example, if two critical alerts have identical severity, the one with higher detection confidence is correctly promoted. This shows that even when a detector is strong, explicitly modeling the relationship between severity and confidence provides a more refined and operationally perfect ranking. Although the numerical gain over the baseline methods is modest in this high-signal environment, the perfect score validates the core principle of our method. In this split, predicted alerts are overwhelmingly attacks; therefore, we emphasize NDCGrel across multiple k cutoffs to separate ranking quality. The overall vs. predicted NDCGrel values differ only for Severity-Only, reflecting benign dominance in the full alert stream. In particular, the Severity-Only and Weighted-Sum methods produce identical scores. This occurs because the
16
high-performance detector assigns uniformly high confidence scores to alerts in the predictedattack queue, causing the confidence term in the weighted sum to have negligible impact on the final rank ordering, which remains dominated by severity. As a limitation, the strong full-feature detector and attack-dominant queue compress ranking differences, so the stress test, confidence-band analysis, and statistical significance assessed in the next subsections provide more discriminative evidence.
5.3
Stress-Test: Reduced-Feature LR
To isolate the effect of detector degradation, we stress the ranking pipeline by training a reducedfeature (flags-only) LR. We restrict the model to weakly discriminative TCP flag features, the LR classifier, faced with heavy class imbalance, defaults toward predicting the benign class. As a result, the alert stream and the confidence scores are both degraded. This changes the alert stream itself and produces a low-recall queue (recall 0.1055; F1 0.1786), so the ranking differences become visible (we separately analyze calibration drift with fixed alert sets in Section 5.7). Table 6 presents the ranking performance under this degraded detector. The results elucidate the disparities in ranking performance. Table 6: Predicted-alert ranking performance under flags-only LR Method
NDCGrel @10pred NDCGrel @50pred NDCGrel @100predNDCGrel @500pred
Severity-Only Confidence-Only Weighted Sum Risk-Averse (κ = 1)
1.0000 0.0040 0.9842 1.0000
1.0000 0.0040 0.9009 0.9984
1.0000 0.0040 0.8215 0.9963
1.0000 0.0041 0.9484 0.9911
Our method scores a near-perfect NDCGrel of 0.9963 under the flags-only stress test. It is particularly striking compared to the baseline methods. To understand this result, consider the ranking task: even with a noisy detector, the true attacks in the predicted-alert queue still have high severity scores (they are real attacks). The Risk-Averse method successfully identifies these high-severity true attacks and ranks them near the top, despite the detector’s low confidence in them. When hi is low, due to its design, the method degrades gracefully rather than catastrophically. The small 0.37% gap from perfect (0.9963 vs. 1.0000) represents only a few of the ranking errors among the top-100 alerts due to minor misrankings between alerts of similar severity. Other remarks: • Severity-Only remains at 1.0 across all k’s because NDCGrel @100 assigns zero relevance to benign FPs, so ranking by ci exactly matches the graded-relevance ordering within the predicted-alert queue. • The catastrophic failure of the Confidence-Only method (NDCGrel @100 of 0.0040) illustrates the danger of relying on a single fragile signal. When the detector becomes unreliable, it leads to a ranking that prioritizes high-confidence FPs. Under stress-test conditions, the detector’s confidence score no longer matches the actual situation. Consequently, ranking by confidence alone prioritizes a stream of high-confidence FPs, burying the few true attacks and rendering the prioritization useless. • Weighted-Sum achieves only 0.8215 at the same cutoff—a 17.85% degradation because its fixed 0.5 weighting cannot adapt to the now-unreliable confidence signal; it continues to pollute the ranking by giving equal importance to a noisy input. 17
The key finding of this stress test is that while simple methods may suffice when detectors are perfect, they fail under realistic conditions. Explicitly modeling uncertainty through the fuzzy framework is a practical necessity to maintain robust alert prioritization in the face of real-world detector degradation. The Risk-Averse method provides near-perfect ranking under strong detectors and shows solid robustness under degraded detectors through its principled treatment of multiple uncertainty sources.
5.4
Confidence-Band Analysis
To isolate where ranking decisions matter most, we partition predicted alerts into three confidence bands (0.3–0.5, 0.5–0.7, 0.7–1.0) and compute NDCGrel @100 within each band. The mid-confidence band (0.5–0.7) is the most diagnostically informative: Confidence-Only drops to 0.3697, while Risk-Averse (κ = 1) remains high at 0.9963. At the broader NDCGrel @500 cutoff, the same mid-band remains stable for Risk-Averse (κ = 1 = 0.9912). Severity-Only and Weighted Sum are near-perfect because confidence varies little within a band. We consider the mid-confidence band as the main way to assess performance, and we include complete band results in the Appendix (Table 11).
5.5
Statistical Significance
To quantify whether the observed differences between methods are statistically meaningful and not due to random chance, we employ a paired bootstrap test on NDCGrel @500 for the CICIDS2017 test set with the full-feature LR detector. We use paired resampling because the same test set is evaluated by all methods, making paired comparisons more statistically powerful than the unpaired test, and we use NDCGrel @500 for bootstrap testing because it balances stability and operational focus: @10 and @100 are too sensitive to small rank swaps (high B variance). Each comparison gives the mean difference ∆ = NDCGA rel - NDCGrel , along with a 95% confidence interval (CI) and a p-value, the probability of observing a mean NDCGrel @500 difference at least as extreme as the one measured. Positive ∆ indicates that Method A ranks higher-severity attacks better; negative ∆ indicates Method B performs better. Table 7: Paired bootstrap results on NDCGrel @500 Method A
Method B
Confidence-Only Severity-Only Weighted Sum
Risk-Averse (κ = 1) Risk-Averse (κ = 1) Risk-Averse (κ = 1)
∆
95% CI
p
-0.2265 0.0016 0.0022
[-0.2555, -0.1909] [-0.0014, 0.0029] [0.0008, 0.0031]
0.0001 0.1620 0.0260
Table 7 shows that Risk-Averse significantly outperforms Confidence-Only; shows no statistically reliable difference versus severity baseline, and almost as good as Weighted Sum with a tiny effect size. These conclusions are stable across @1,000, @5,000, and @10,000 bootstrap resamples.
5.6
Auxiliary Detector Families
To assess sensitivity to the detector family, we ran auxiliary experiments with a supervised Random Forest and an unsupervised Isolation Forest. The Random Forest backend yields near-perfect rankings for severity-based methods because the detector is strong and confidence scores saturate; the comparison therefore collapses, as expected. The Isolation Forest backend highlights a different failure mode: its anomaly scores are not calibrated as attack probabilities, so Confidence-Only collapses even after normalization, while Risk-Averse remains stable because it continues to anchor rankings on severity. Full results are shown in Appendix A.5. 18
5.7
Robustness Under Detector Miscalibration
Real-world detectors often become miscalibrated due to domain shift (when network traffic patterns change), seasonal variations (when attack patterns change over time), and training data drift (when the historical data used to train the detector no longer reflect current conditions). Unlike the stress test, which changes the detector and thus the alert stream, in this section we isolate the calibration drift by keeping the alert set fixed and perturbing only the confidence scores pi . This let us measure how sensitive each ranking method is to miscalibration without conflating the effect with changes in detection coverage. We test the following three scenarios: Scenario 1: Optimistic Bias (Detector Overconfident) – pi × 1.15: simulates a detector that systematically overestimates its confidence. This happens when a detector is trained on past data, but encounters a new attack distribution with slightly different characteristics. We simulate this by scaling confidence scores: p′i = min(1.0, pi × 1.15), capping at 1.0 to maintain validity. The results show minimal degradation across all methods: • Severity-Only: 0.00% • Confidence-Only: −0.13% • Weighted Sum: −1.53% • Risk-Averse: 0.00% The key insight is that Risk-Averse’s height-capping mechanism (hi ≤ 1.0) naturally bounds the effect of overconfidence. When pi is scaled up, the height hi saturates at 1.0, limiting the damage to the ranking. Conversely, the Weighted Sum’s constant 0.5 weighting permits the inflated confidence to distort the ranking, resulting in a 1.53% decline. Consequently, this illustrates that Risk-Averse’s fuzzy framework inherently offers resilience against systematic overconfidence. Scenario 2: Pessimistic Bias (Detector Underconfident) – pi × 0.85: simulates a uniformly conservative detector by scaling all confidence scores down: p′i = pi × 0.85. Uniform scaling preserves ordering (0.00% change for all methods). This finding validates that our normalization strategy, which scales confidence to match CVSS range, successfully absorbs uniform calibration bias. Scenario 3: Random Drift (+0.2 Noise) – pi + N (0, 0.2): simulates stochastic noise in confidence scores, representing random fluctuations due to detector instability, network variability, or measurement error. We add a Gaussian noise: p′i = clip(pi + N (0, 0.2), 0, 1). The results show a clear differentiation between methods: noise harms Confidence-Based methods and Risk-Averse remains stable because the logarithmic term reduces the effect of noise, and the uncertainty term σi provides a damping effect. • Severity-Only: 0.00% • Confidence-Only: −2.93% • Weighted Sum: −0.71% • Risk-Averse: +1.92% Across all scenarios, Risk-Averse is at least as robust as the baselines. Height-capping bounds overconfidence, normalization absorbs uniform bias, and log-scaling with uncertainty damps stochastic noise. 19
6
Conclusion
Alert fatigue represents a significant impediment within security operations, as analysts are inundated with numerous daily alerts, the majority of which are FPs. This research tackles this issue by introducing a framework designed for alert prioritization. The framework explicitly incorporates three sources of uncertainty: threat severity, detection confidence, and organizational risk attitude. Our key contributions include: (1) introducing subnormal Gaussian fuzzy numbers to IDS alert prioritization, enabling principled uncertainty modeling, (2) providing a novel framework that allows organizations to tune their security posture through the risk-attitude parameter κ, (3) validating that our framework is near-parity with baselines on strong detectors but substantially robust when detectors degrade with different attack distributions on CIC-IDS2017 and NSL-KDD datasets, (4) showing that Risk-Averse maintains 0.9963 NDCGrel @100 while alternatives degrade to 0.0040–0.8215, achieving greater robustness, and (5) revealing that the mid-confidence band (0.5–0.7) is where explicit uncertainty modeling provides the greatest value. In practice, such as in a typical SOC with 10,000 alerts/day and 80% FPs, even a 10% improvement in ranking quality frees analysts to focus on critical threats. Our method achieves this while maintaining robustness to realistic detector degradation. Statistical significance is validated through paired bootstrap testing (p < 0.05), and robustness is confirmed across detector families (Logistic Regression, Random Forest, Isolation Forest) and realistic miscalibration scenarios (overconfidence, underconfidence, stochastic noise). Regarding limitations and future research: we assume CVSS scores are available for all attacks and novel attacks may lack scores, thus evaluation on public datasets should be supplemented with real-world SIEM data. In addition, parameter κ is fixed; therefore, future work should develop adaptive learning methods for this parameter. Future work will also incorporate temporal dynamics to capture evolving attack patterns and detector performance drift, enabling adaptive ranking under non-stationary conditions.
A
Supplementary Tables
A.1
Experimental Protocol
This section provides a detailed, step-by-step recipe that shows how we conducted our experiments for the sake of transparency, validation, and reproducibility. Having prepared our dataset, the phases below are followed. A.1.1
Training Phase
In this phase, we: 1. train the baseline IDS (LR) on the training set (Mon/Tue data) with solver saga, C = 1.0, max iterations = 1000, and class weights set to balanced, 2. if the day-based split is attack-sparse (<5%), fall back to a stratified 50/20/30 split 3. calibrate predicted probabilities on the validation set using a sigmoid calibrator and evaluate the baseline IDS on the validation set (Wed data), 4. for each attack class j compute detection confidence metrics (Precision, Recall, F1) for each attack class, and 5. derive height values hj from the smoothed F1 and store them in a lookup table to use during alert ranking.
20
A.1.2
Testing Phase
In this phase, for each test set (Thu/Fri data), we: 1. run the baseline IDS on the test set to generate alerts, 2. for each alert, extract the attack type and raw IDS confidence, 3. for each ranking method (Severity-Only, Confidence-Only, Weighted Sum, Risk-Averse): (a) compute the ranking score for each alert; (b) sort alerts by ranking score; (c) compute evaluation metrics 4. compare the ranking methods across all metrics A.1.3
Parameter Tuning
We sweep κ ∈ {0, 0.5, 1, 1.5, 2} and present the sensitivity analysis in Appendix A.2; main results use κ = 1 as a standard setting rather than tuning per dataset. A.1.4
Statistical Significance Testing
To quantify whether differences between methods are statistically meaningful, we perform a paired bootstrap test on NDCGrel @500 computed from the top-500 ranked alerts: 1. For each test set, take the top-500 alerts from each method (based on rank) and form graded relevance vectors 2. Compute NDCGrel @500 for each method and bootstrap the mean difference using paired resampling over rank positions 3. Report bootstrap p-values and 95% confidence intervals for the mean difference in NDCGrel @500 4. Consider the differences significant at p < 0.05
A.2
Height Parameter and Risk-Attitude Sensitivity
In the main text, we present results with the default setting κ = 1; here we provide sensitivity examples to show how rankings shift under higher risk aversion. To demonstrate the value of the height parameter (detection confidence), we used three real DoS alerts from the CICIDS2017 test set with low, medium, and high heights by default κ = 1. The low-height alert (alert_id 353856, hi = 0.3796) is ranked lower than the medium and high-height alerts by the method, while the confidence-only baseline places it much lower overall. Table 8 also lists the corresponding severity core ci and spread σi values. Table 8: Impact of Height Parameter on Alert Ranking (CIC-IDS2017 test set) Alert ID
Type
hi
pi
ci
σi
RiskAverse Rank
ConfidenceOnly Rank
353856 192641 230833
DoS DoS DoS
0.3796 0.7992 0.7992
0.3796 0.9872 1.0000
7.2504 7.3872 7.8000
1.4706 1.4267 1.2480
24463 5195 403
179313 60304 1345
21
When an alert has low detection confidence, our method deprioritizes it (rank 24463); a confidence-only baseline, driven solely by pi , places it even lower (rank 179313). This demonstrates the value of explicitly modeling detection confidence through the height parameter. We also show κ sensitivity for three DoS alerts with low, medium, and high heights to show how the risk-attitude parameter shifts rankings. As κ increases, the low-height alert is penalized more, while medium- and high-height alerts shift only modestly (Table 9). This flexibility allows organizations to tune their security posture: aggressive organizations can use κ = 0 to focus on high-severity threats, while conservative organizations can use κ ≥ 1 to focus on high-confidence threats. Table 9: Impact of Risk-Attitude Parameter κ on Alert Ranking. As κ increases, the ranking shifts. Specifically, the alert with the highest severity but lowest confidence (Rank 9467 when κ=0) is moved down the list. At the same time, the alert with lower severity but higher confidence (Rank 403 when κ=1) is moved up. This change illustrates the trade-off between severity and confidence. Alert ID
Type
353856 192641 230833
DoS low hi DoS med hi DoS high hi
A.3
hi
ci
σi
κ=0
κ=1
κ=2
0.3796 0.7992 0.7992
7.2504 7.3872 7.8000
1.4706 1.4267 1.2480
Rank: 9467 Rank: 7436 Rank: 946
Rank: 24463 Rank: 5195 Rank: 403
Rank: 41807 Rank: 5426 Rank: 675
Contextual Criticality (Heartbleed)
To show that more critical instances of the same attack type are prioritized more, we use two Heartbleed alerts from the test set with different contextual factors under the default setting κ = 1. The higher-criticality alert (larger cfi ) is ranked far above the lower-criticality alert, indicating that contextual criticality remains decisive even when risk adjustment is applied (Table 10). Table 10: Contextual criticality effect for Heartbleed alerts (CIC-IDS2017 test set, κ = 1) Alert ID
Type
295490 57807
Heartbleed Heartbleed
A.4
cfi
ci
hi
pi
Risk-Averse Rank
0.2242 0.9838
2.1792 9.8384
0.0002 0.5000
0.0002 0.9575
849172 13886
Confidence-Band Analysis
Table 11 provides the full three-band breakdown for the flags-only stress test with κ = 1. The mid-confidence band (0.5–0.7) is the most discriminative region, while the low- and highconfidence bands show compressed differences. This table complements the main-text focus on mid-confidence alerts as the primary diagnostic view.
A.5
Auxiliary Detector Families
We present auxiliary results for a supervised Random Forest and an unsupervised Isolation Forest to probe detector-family sensitivity. Each cell includes NDCGrel @10pred / NDCGrel @100pred for the predicted-alert queue. Random Forest is a strong supervised backend, so severity-based rankings are near-perfect. Isolation Forest uses anomaly scores that are not calibrated probabilities, so Confidence-Only performs poorly, while Risk-Averse remains stable by anchoring on severity. 22
Table 11: Confidence-band NDCGrel @100 under flags-only LR Method
NDCGrel @100 NDCGrel @100 NDCGrel @100 (0.3–0.5) (0.5–0.7) (0.7–1.0)
Severity-Only Confidence-Only Weighted Sum Risk-Averse (κ = 1)
0.9802 0.2692 0.9802 0.9982
1.0000 0.3697 1.0000 0.9963
1.0000 0.0053 1.0000 0.9601
Table 12: Auxiliary detector results (NDCGrel @10pred / NDCGrel @100pred ) DetectorSetting
SeverityOnly
RF
1.0000 1.0000 1.0000 1.0000 0.8689 0.9715 0.8689 0.9715
RF IF IF
A.6
Full features Flags-only Full features Flags-only
/ / / /
Confidence- Weighted Only Sum
RiskAverse (κ = 1)
0.5794 0.5658 0.3697 0.3697 0.0000 0.0000 0.0223 0.0223
1.0000 1.0000 1.0000 0.9963 1.0000 1.0000 1.0000 1.0000
/ / / /
1.0000 1.0000 1.0000 1.0000 0.9029 0.9732 0.9676 0.9855
/ / / /
/ / / /
Parameter Sensitivity
We varied height-smoothing parameters and globally scaled uncertainty factors to evaluate sensitivity of Risk-Averse ranking (full-feature LR, CIC-IDS2017). Table 13 shows that NDCGrel @10pred stays within 0.9992–0.9995 and NDCGrel @100pred within 0.9862–0.9916 across all settings. The largest shifts occur under global scaling of uncertainty factors; α, hmin , and hmax have negligible effect in this range.
A.7
Multi-Dataset Validation: NSL-KDD
We also evaluated the framework on NSL-KDD [30] as a generalization check across datasets with different attack distributions. Using the full-feature LR detector, ranking methods are nearidentical, consistent with the CIC-IDS2017 pattern where strong detectors compress differences. This supports the view that meaningful separation appears primarily when the alert stream is noisy or uncertain.
B
Implementation Details
B.1
Software and Libraries
• Language: Python 3.8+ • Machine Learning: scikit-learn 1.0.2 (Logistic Regression, metrics) • Data Processing: pandas 1.3.5, numpy 1.21.6 • Visualization: matplotlib 3.5.1, seaborn 0.11.2 • Statistical Testing: numpy/scipy (paired bootstrap) 23
Table 13: Risk-Averse sensitivity to height smoothing and uncertainty scaling (NDCGrel @10pred / NDCGrel @100pred ). Parameter
Value
NDCGrel @10pred
NDCGrel @100pred
α α α α hmin hmin hmin hmax hmax hmax ufi scale ufi scale ufi scale
0.50 0.70 0.90 0.95 0.01 0.05 0.10 0.90 0.95 0.99 0.80× 1.00× 1.20×
0.9994 0.9994 0.9994 0.9994 0.9994 0.9994 0.9994 0.9994 0.9994 0.9994 0.9995 0.9994 0.9992
0.9914 0.9890 0.9890 0.9890 0.9890 0.9890 0.9890 0.9890 0.9890 0.9890 0.9916 0.9890 0.9862
B.2
Experiment Configuration
• Baseline IDS: Logistic Regression (solver saga, C = 1.0, max iterations = 1000, class weights balanced) with sigmoid probability calibration. • Risk Parameters: κ ∈ {0, 0.5, 1, 1.5, 2}; height smoothing α = 0.9, hmin = 0.05, hmax = 0.95; instance height hi = min(hclass , pi ). • Stress Test: flags-only feature subset (columns containing “flag”), minimum 5 features. • Confidence Bands: [0.3, 0.5), [0.5, 0.7), [0.7, 1.0]. • Random Seed: seed=42 for numpy/python RNG and data splitting. • Auxiliary Detectors: Random Forest (supervised) and Isolation Forest (unsupervised) runs for detector-family sensitivity, shown in Appendix A.5.
B.3
Reproducibility
To ensure reproducibility: • Random seeds are fixed (seed=42) for data splits and RNG-dependent steps • Dataset preprocessing and split logic (including CIC-IDS2017 fallback when the train split has < 5% attacks) is documented • Hyperparameters for the baseline IDS are specified • Attack-type mapping is centralized with an optional override CSV • Results, plots, and dataset profiles are saved in results/ folder
References [1] Xiaoyu Wang, Xueping Liang, Xiu Zhang, Wei Zhang, and Xiaorui Gong. Combating alert fatigue with alertpro: Context-aware alert prioritization using reinforcement learning for 24
multi-step attack detection. Computers & Security, 137:103583, 2024. ISSN 0167-4048. doi: 10.1016/j.cose.2023.103583. URL https://www.sciencedirect.com/science/article/ pii/S0167404823004935. [2] Fatemeh Jalalvand, Mohan Baruwal Chhetri, Surya Nepal, and Cecile Paris. Alert prioritisation in security operations centres: A systematic survey on criteria and methods. ACM Computing Surveys, 57(2):42:1–42:36, 2025. doi: 10.1145/3695462. [3] Bushra A. Alahmadi, Louise Axon, and Ivan Martinovic. 99% false positives: A qualitative study of soc analysts’ perspectives on security alarms. In Proceedings of the 31st USENIX Security Symposium (USENIX Security 22), pages 1–18. USENIX Association, 2022. [4] SANS Institute. Sans 2024 detection and response survey: Transforming cybersecurity operations. Technical report, SANS Institute, 2024. URL https://www.sans.org/ white-papers/sans-2024-detection-response-survey. [5] Robin Sommer and Vern Paxson. Outside the closed world: On using machine learning for network intrusion detection. In IEEE Symposium on Security and Privacy, pages 305–316. IEEE, 2010. doi: 10.1109/SP.2010.25. [6] Martin Roesch. Snort: Lightweight intrusion detection for networks. In Proceedings of the 13th USENIX Conference on System Administration, pages 229–238. USENIX Association, 1999. [7] Richard P. Lippmann, David J. Fried, Ian Graf, and et al. Evaluating intrusion detection systems: The 1998 darpa off-line intrusion detection evaluation. In Proceedings of the DARPA Information Survivability Conference and Exposition, volume 2, pages 12–26. IEEE, 2000. [8] Leo Breiman. Random forests. Machine Learning, 45:5–32, 2001. [9] Ahmad Javaid, Quamar Niyaz, Weiqing Sun, and Mansoor Alam. A deep learning approach for network intrusion detection system. In 9th EAI International Conference on Bio-inspired Information and Communications Technologies (formerly BIONETICS). EAI, 2016. doi: 10.4108/eai.3-12-2015.2262516. [10] R. C. Staudemeyer and E. R. Morris. Understanding lstm: a tutorial into long short-term memory recurrent neural networks, 2019. URL https://arxiv.org/abs/1909.09586. [11] Klaus Julisch. Clustering intrusion detection alarms to support root cause analysis. ACM Transactions on Information and System Security, 6:443–471, 2003. [12] J. A. Wong, A. M. Berenbeim, D. A. Bierbrauer, and N. D. Bastian. Uncertainty-quantified, robust deep learning for network intrusion detection. In Winter Simulation Conference, pages 1–12. IEEE, 2023. [13] Pankaj Malhotra et al. Lstm-based encoder-decoder for multi-sensor anomaly detection, 2016. [14] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, pages 1321–1330. PMLR, 2017. [15] Marco T. Ribeiro, Sameer Singh, and Carlos Guestrin. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144. ACM, 2016. 25
[16] Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73:1–15, 2018. doi: 10.1016/j.dsp.2017.10.011. [17] Hans-Jürgen Zimmermann. Fuzzy Set Theory and Its Applications. Springer, 4 edition, 2001. doi: 10.1007/978-94-010-0646-0. [18] Peter Mell, Karen Scarfone, and Sasha Romanosky. A complete guide to the common vulnerability scoring system version 2.0. Technical Report 800-145, NIST, 2007. [19] National Vulnerability Database. Common vulnerability scoring system version 3.0 specification. Technical report, FIRST, 2015. [20] Luca Allodi and Fabio Massacci. Comparing vulnerability severity and exploits using case-control studies. In Proceedings of the 2012 ACM Conference on Computer and Communications Security, pages 386–397. ACM, 2012. [21] Jay Jacobs, Sasha Romanosky, and Benjamin Edwards. Improving vulnerability remediation through better exploit prediction. In IEEE Symposium on Security and Privacy, pages 1–16. IEEE, 2019. [22] Chi-Ho Tsang, Sam Kwong, and Hanli Wang. Genetic-fuzzy rule mining approach and evaluation of feature selection techniques for anomaly intrusion detection. Pattern Recognition, 40(9):2373–2391, 2007. doi: 10.1016/j.patcog.2006.12.009. [23] A. K. Kar and S. Mishra. A fuzzy-based approach for detecting http-based attacks. In 2012 International Conference on Advances in Computing, Communications and Informatics, pages 1–6. IEEE, 2012. [24] Weizhi Meng, Wenjuan Li, Lam-For Kwok, and Kim-Kwang Raymond Choo. Enhancing cloud-based intrusion detection systems with log-based alert verification. IEEE Transactions on Cloud Computing, 3:242–254, 2013. [25] S.-H. Chen. Ranking fuzzy numbers with maximizing set and minimizing set. Fuzzy Sets and Systems, 17(2):113–129, 1985. doi: 10.1016/0165-0114(85)90050-8. [26] L.A. Zadeh. Fuzzy sets. Information and Control, 8(3):338–353, 1965. ISSN 0019-9958. doi: https://doi.org/10.1016/S0019-9958(65)90241-X. URL https://www.sciencedirect. com/science/article/pii/S001999586590241X. [27] H. G. Akdemir and M. Moran. Shortest path problem with subnormal gaussian fuzzy costs. Submitted, 2025. [28] Iman Sharafaldin, Arash Habibi Lashkari, and Ali A. Ghorbani. Toward generating a new intrusion detection dataset and intrusion traffic characterization. In International Conference on Information Systems Security and Privacy (ICISSP), pages 108–116. SciTePress, 2018. doi: 10.5220/0006639801080116. [29] Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems, 20(4):422–446, 2002. doi: 10.1145/582415. 582418. [30] Mahbod Tavallaee, Ebrahim Bagheri, Wei Lu, and Ali A. Ghorbani. A detailed analysis of the kdd cup 99 data set. In IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), pages 1–6. IEEE, 2009. doi: 10.1109/CISDA.2009. 5356528. 26