Too Sure to Be Safe: Model Calibration for Reliable Log Anomaly Detection

Too Sure to Be Safe: Model Calibration for Reliable Log Anomaly Detection Bin Li1∗ , Dongdong Wang2∗ , Siyang Lu1† 1

Beijing Jiaotong University, Beijing, China University of Florida, Gainesville, FL, USA 24120434@bjtu.edu.cn, dongdongwang@ufl.edu, sylu@bjtu.edu.cn

arXiv:2608.17965v1 [cs.LG] 18 Aug 2026

2

heavily on high-quality annotations and reliable preprocessing pipelines. In this paper, we focus on supervised classificationbased log anomaly detectors and investigate the reliability of their confidence estimation. However, strong classification performance does not guarantee reliable decision-making. In log anomaly detection, prediction errors are asymmetric in their operational impact [8]. A normal log sequence that is incorrectly classified as anomalous may trigger a false alarm, causing additional inspection costs and potentially contributing to alert fatigue. While such errors are undesirable, they still expose suspicious cases to operators or downstream monitoring modules. By contrast, an anomalous log sequence that is incorrectly classified as normal constitutes a missed anomaly, allowing the underlying abnormal behavior to bypass the detection pipeline. This risk becomes more severe when the detector assigns high confidence to the incorrect normal prediction, as the system may treat the decision as reliable and suppress further intervention. Therefore, highconfidence false negatives constitute a central reliability risk for supervised log anomaly detectors. We first examine the confidence behavior of representative supervised log anomaly detectors. The results reveal a recurring reliability issue: detectors can be confidently wrong. Both false I. I NTRODUCTION positives and false negatives often receive confidence scores Large-scale data systems are increasingly critical in the comparable to correctly classified samples, suggesting that era of artificial intelligence and high-performance computing. detector confidence is not consistently aligned with prediction Maintaining system reliability relies heavily on continuous correctness. This issue is not simply resolved by increasing log monitoring, making log anomaly detection essential for model complexity. More complex detectors may obtain higher identifying abnormal events and preventing system failures. accuracy and more favorable aggregate calibration scores, yet Existing log anomaly detection approaches have achieved their confidence distributions often remain highly concentrated substantial progress in semantic log analysis and anomalous near one, leaving incorrect predictions overconfident. We also event detection [1]–[4]. find that anomalous samples tend to exhibit larger calibration Current methods for log anomaly detection can generally be errors than normal samples. These observations indicate that categorized into two paradigms [5]. The first models normal strong benchmark performance can still conceal important log patterns and identifies deviations from expected behavior reliability risks, particularly when anomalous log sequences as anomalies. The other formulates log anomaly detection are confidently predicted as normal. as a supervised classification problem, where models directly To mitigate this risk, we design a selector-based calibration predict whether a log sequence is normal or anomalous. While framework for supervised log anomaly detectors. Given the the former typically requires less labeled data, the latter often output of a pre-trained log anomaly detector, the selector achieves stronger benchmark [6], [7] performance but relies predicts whether the detector’s current prediction is correct or incorrect. Specifically, for each input, we use the detector’s Accepted at the 2026 IEEE International Conference on Data Mining. ∗ These authors contributed equally to this work. hidden representation as the basis for reliability estimation. † Corresponding author: Siyang Lu (sylu@bjtu.edu.cn). Since predictions assigned to the normal class and those

Abstract—Online log anomaly detection is critical for maintaining the reliability of large-scale computing systems. Although recent language model-based log anomaly detectors achieve strong detection performance, their confidence estimates remain poorly calibrated. We show that these detectors frequently assign excessive confidence to incorrect predictions, particularly for anomalous logs under severe class imbalance. Moreover, confidence on erroneous predictions remains persistently high even when conventional calibration metrics indicate good calibration, creating a critical reliability gap for operational monitoring systems. To address this issue, we propose Log Reconstruction and Distance (LoRD), a lightweight post-hoc calibration framework for reliable log anomaly detection. LoRD learns predictionroute-specific reliability models from latent representations of correctly classified validation samples and estimates prediction reliability through route-wise reconstruction distances. Based on the estimated reliability, LoRD selectively recalibrates high-risk predictions to suppress overconfident errors while preserving reliable predictions. Extensive experiments on four large-scale log benchmark datasets and multiple language model-based detectors demonstrate that LoRD consistently improves confidence reliability and substantially reduces overconfident anomaly-related errors without sacrificing anomaly detection performance. Index Terms—Log anomaly detection, model calibration, confidence reliability, overconfidence

assigned to the anomalous class exhibit different error patterns, we construct two prediction routes according to the detector’s output label. For each route, an autoencoder is trained on the hidden representations of correctly classified validation samples, and its reconstruction error is used as a reliability score. Although the two routes follow the same autoencoder-based scoring principle, their calibration objectives are asymmetric. The predicted-normal route prioritizes potential false negatives, while the predicted-anomalous route is treated more conservatively to avoid weakening reliable anomaly predictions. This design reflects the asymmetric risk of log anomaly detection, where missed anomalies are typically more critical than false alarms. In this work, we first show that supervised log anomaly detectors can remain overconfident on misclassified samples despite strong benchmark performance. We then propose a selector-based calibration framework that predicts whether a detector’s output is correct or incorrect using its predictions and hidden representations. Finally, we demonstrate across multiple datasets and representative detectors that our method achieves a task-oriented calibration trade-off, yielding more reliable confidence estimates for anomalous samples, especially those confidently predicted as normal, while keeping the calibration degradation on normal samples within an acceptable range. Our contributions are summarized as follows: • To the best of our knowledge, this is the first work to systematically investigate reliability and confidence calibration in language model-based log anomaly detection. We reveal a persistent overconfidence phenomenon in which state-of-the-art detectors assign excessively high confidence to erroneous anomaly predictions despite exhibiting strong performance under conventional calibration metrics. Through extensive analyses across history window length, LoRA rank, model complexity, and classimbalance mitigation strategies, we show that Confidence on Error (CoE) remains consistently high across models and settings, exposing a critical reliability gap that is overlooked by existing calibration metrics. • We propose LoRD (Log Reconstruction and Distance), a lightweight post-hoc calibration framework for log anomaly detection. LoRD learns route-specific reconstruction models that characterize the reliable conditional latent distributions of normal and anomalous predictions, and leverages reconstruction distances as reliability signals for tri-region confidence calibration. By selectively preserving reliable predictions while suppressing overconfident errors, LoRD directly targets the reliability challenge identified in modern log anomaly detectors. • We conduct extensive experiments on four large-scale log benchmark datasets using diverse language modelbased log anomaly detectors with varying architectures and model scales. Experimental results demonstrate that LoRD consistently improves reliability and calibration quality across datasets and detectors, effectively reducing overconfident mispredictions while preserving detection performance. These findings highlight the robustness,

effectiveness, and generalizability of LoRD for reliabilityaware log anomaly detection. II. R ELATED W ORK A. Log Anomaly Detection Researchers have extensively employed sequence modeling techniques for system log analysis with notable success. Among these applications, log anomaly detection has emerged as a critical task, often formulated as a text classification problem in natural language processing. Driven by advances in deep neural networks, increasingly powerful text classification models have substantially improved the accuracy and robustness of log anomaly detection. Log anomaly detection aims to identify abnormal system behaviors from log data. Existing studies mainly follow two lines [5]: i) semi-supervised methods and ii) supervised methods. Semi-supervised methods usually learn normal log patterns from historical logs and identify samples that deviate from the learned normality as anomalies. DeepLog [9] is a representative method in this line, which models normal log sequences with an LSTM and detects anomalies based on deviations from predicted execution patterns. LogAnomaly [10] extends this idea by considering both sequential and quantitative anomalies, and further incorporates semantic representations of log templates to better handle unstructured logs. PLELog [11] reduces the reliance on manual labeling by estimating probabilistic labels and learning semantic representations for log anomaly detection. Supervised methods treat log anomaly detection as a binary classification task, learning discriminative patterns from labeled normal and anomalous sequences. Inspired by TextCNN, Lu et al. [12] apply multiple convolutional filters with different kernel sizes in parallel to capture local patterns at different granularities, achieving strong performance in supervised log anomaly detection. LogRobust [13] combines semantic log-template representations with an attention-based recurrent model to improve robustness to unstable log events. LightLog [14] improves efficiency by applying PCA-based dimensionality reduction to log semantic representations before using a lightweight temporal convolutional network to model sequential dependencies. NeuralLog [15] uses pretrained language models to encode the semantics of log events and adopts a Transformerbased encoder to capture contextual dependencies within log sequences, thereby improving anomaly classification. Recent studies have also begun to explore the potential of large language models in log anomaly detection. LogLLM [16] follows a semantic-enhanced framework similar to NeuralLog: it first uses pretrained language models such as BERT to encode log sequences, then employs a projector to align the extracted log representations with the input space of LLaMA, and finally applies parameter-efficient fine-tuning techniques such as LoRA for anomaly classification. Another line of work adapts GPT-2 [17] to log anomaly detection through a simpler preprocessing pipeline, directly feeding the processed log-text sequences into GPT-2.

Fig. 1. Overview of existing log anomaly detection frameworks. Logs are preprocessed, embedded into semantic representations, encoded using sequential models such as Bi-LSTM or Transformer, and finally used for anomaly prediction.

0.4

0.85

Perf./Conf.

0.8 0.90

Calib. Error

0.95

1.0

2.0

0.9

1.5

0.8

1.0

0.7

Varying Model Size

10

20 30 History Length

Imbalance Handling

0.002

0.950 0.925

0.001

0.900

8 16 LoRA Rank

(a)

0.90

0.08

0.85 0.80

0.04

0.75

0.0 4

0.12

0.95

0.875

50

1.00

0.003

0.975

0.5

0.0 5

CoE

1.000

0.6 0.80

Brier

Calib. Error Conf.

1.2

NLL

Calib. Error

ECE

Varying LoRA Rank

1.00

Perf./Conf.

CoC

Perf./Conf.

Varying History Length

F1

Calib. Error

Acc

0.000

32

1.67M

1.70M 1.71M Model Size

(b)

0.70

2.67M

0.00 None

Label Sm.

Over Samp.

(c)

Under Samp.

Bal. Samp.

Class Wtd.

Focal Loss

(d)

Fig. 2. Ablation analysis of factors affecting confidence calibration and error confidence in log anomaly detection. Panels (a)–(c) report Acc, F1, and CoE on the left axis and ECE, NLL, and Brier score on the right axis. Panel (d) reports abnormal-class ECE, Brier score, CoC, and CoE for imbalance handling strategies. Green bars denote higher-is-better metrics, while red bars denote lower-is-better metrics. Label Sm., Over Samp., Under Samp., Bal. Samp., and Class Wtd. denote Label Smoothing, Over-Sampler, Under-Sampler, Balanced Sampler, and Class-Weighted Loss, respectively. Normal

10

10

10

−3

Nor. CoE

Abn. CoE

Brier: Normal vs Abnormal 10

0

Confidence by Class 1.00

−1

0.95

Brier (log)

10

−2

Abn. CoC

NLL: Normal vs Abnormal

−1

NLL (log)

ECE (log)

10

Nor. CoC

−1

10

Confidence

ECE: Normal vs Abnormal 10

Abnormal

−2

0.85

−2

10

−3

0.75 Text CNN

Log Robust

Light Log

Neural Log

(a)

GPT2

Text CNN

Log Robust

Light Log

Neural Log

GPT2

(b)

Text CNN

Log Robust

Light Log

(c)

Neural Log

GPT2

Text CNN

Log Robust

Light Log

Neural Log

GPT2

(d)

Fig. 3. Class-wise calibration comparison on the BGL dataset. Panels (a)–(c) compare normal-class and abnormal-class ECE, NLL, and Brier score using a logarithmic axis. Panel (d) compares class-wise confidence on correct predictions and erroneous predictions.

B. Model Calibration Model calibration aims to make predictive confidence better reflect empirical correctness. Existing methods can be broadly grouped into three categories: regularization-based training, uncertainty estimation, and post-hoc calibration [18]. Regularization-based methods improve calibration by reducing overconfidence during training. Typical examples include label smoothing [19], confidence penalty [20], mixup [21], focal loss [22], and class-balanced loss [23]. These methods can improve calibration, but require detector retraining and

may alter the original detection behavior. Uncertainty estimation methods quantify predictive uncertainty through approaches such as Monte Carlo dropout [24], deep ensembles [25], and Bayesian neural networks [26]. However, they often require extra computation or multiple forward passes, making them less suitable for lightweight posthoc calibration. Post-hoc calibration methods adjust the output probabilities of a trained model without modifying its parameters. Representative methods include Temperature Scaling and Logistic Scal-

ing [27], Beta Scaling [28], and Selective Scaling [29]. These methods improve confidence estimation through probabilitylevel mappings or auxiliary calibration models, and serve as strong post-hoc calibration baselines. III. P RELIMINARIES A. Problem Formulation We study supervised log anomaly detection. The problem is formulated as a binary confidence calibration problem. Given an input log instance xi with binary indicator label yi ∈ {0, 1}, where yi = 0 denotes a normal sample and yi = 1 denotes an anomalous sample, a detector fθ produces a scalar anomaly confidence score: pi = σ(fθ (xi )),

pi ∈ (0, 1),

(1)

where σ(·) denotes the sigmoid function and pi represents the predicted confidence that xi is anomalous. The final prediction is obtained by applying a decision threshold τ : ( 1, if pi ≥ τ, ŷi = (2) 0, if pi < τ. Based on the relationship between the predicted label ŷi and the ground-truth label yi , predictions are partitioned into four categories: true negatives (TN), false positives (FP), true positives (TP), and false negatives (FN). This formulation enables systematic analysis of prediction confidence and calibration behavior across correct and incorrect anomaly predictions.

Unlike ECE, which evaluates calibration by comparing confidence with empirical accuracy, BS measures overall probability quality by directly penalizing probability estimation errors. As a result, BS reflects both calibration and class discrimination. However, similar to ECE, BS aggregates over all predictions and may therefore overlook confidence behavior on rare but critical mispredictions. Confidence on Error (CoE). CoE measures the average confidence assigned to misclassified samples: 1 X CoE = ci , (5) |E| i∈E

where E = {i | ŷi ̸= yi } denotes the set of incorrectly classified samples and ci is the confidence score of sample i. A high CoE indicates that the model is systematically overconfident on its mistakes — a particularly concerning failure mode in anomaly detection, where missed abnormal cases carry high operational cost. Unlike ECE and BS, which aggregate over all predictions, CoE isolates confidence behavior specifically on failure cases, providing a targeted diagnostic that the other metrics do not directly expose. Conversely, Confidence on Correct (CoC) measures the average confidence assigned to correctly classified samples. IV. E MPIRICAL A NALYSIS OF C ONFIDENCE C ALIBRATION IN L OG A NOMALY D ETECTION A. History Length

We analyze the influence of sliding-window history size while fixing the stride to 1. As shown in Figure 2(a), increasing B. Calibration Metrics the history length produces only modest variations in predictive Besides accuracy and F1 score, we evaluate model reliability performance, with Acc remaining above 98% and F1 remaining using several widely adopted confidence calibration metrics for above 0.84 across all settings. Although ECE and Brier score log anomaly detection. For example, negative log-likelihood exhibit noticeable changes as the context length varies, CoE (NLL) evaluates the quality of probabilistic predictions by remains consistently high, ranging from 0.90 to 0.98. Interestmeasuring the likelihood assigned to the ground-truth labels. ingly, larger history windows reduce CoE more substantially Lower NLL indicates better calibrated confidence estimates. To than they improve conventional calibration metrics. While ECE provide a comprehensive assessment of calibration quality, we increases from 0.13 to 0.17 as the history length grows from further consider the following metrics that capture confidence– 5 to 30, CoE decreases from 0.98 to 0.90. This discrepancy accuracy alignment, probabilistic prediction quality, and confi- suggests that average calibration metrics do not adequately reflect confidence behavior on erroneous predictions. Even dence behavior on both correct and incorrect predictions. when additional contextual information improves reliability, Expected Calibration Error (ECE). ECE [30] measures the the model continues to assign excessively high confidence to discrepancy between prediction confidence and empirical accuincorrect predictions. Therefore, longer temporal context alone racy by partitioning predictions into M fixed-width confidence is insufficient to resolve the overconfidence problem. bins: ECE =

M X |Bm | m=1

N

|acc(Bm ) − conf(Bm )| ,

(3)

where Bm denotes the m-th confidence bin, N is the total number of samples, and acc(Bm ) and conf(Bm ) represent the empirical accuracy and average confidence within the bin, respectively. Brier Score (BS). BS measures the mean squared error between predicted probabilities and ground-truth labels: N

BS =

1 X (pi − yi )2 . N i=1

(4)

B. LoRA Rank Figure 2(b) presents the impact of LoRA rank on calibration behavior. Increasing the LoRA rank substantially improves predictive performance, with F1 increasing from 0.61 at rank 4 to 0.94 at rank 32. Accuracy exhibits a similar trend, indicating that larger adaptation capacity enables the model to capture richer task-specific patterns. However, calibrationrelated metrics remain largely unchanged. ECE and Brier score stay nearly constant across all LoRA ranks, while CoE remains extremely high, approaching 1.0 regardless of adaptation capacity. These results suggest that increasing model

expressiveness primarily improves predictive accuracy but receive high confidence, incorrectly classified abnormal samples provides limited benefit for mitigating overconfident errors. (Abn. CoE) also receive surprisingly high confidence scores, Even highly accurate models continue to assign near-maximal often exceeding 0.9. In contrast, confidence on incorrectly confidence to misclassified samples. This observation further classified normal samples (Nor. CoE) remains substantially demonstrates that improving predictive performance alone is lower and closer to the uncertainty region. This indicates insufficient for reliable confidence estimation. that language model-based log anomaly detectors tend to be particularly overconfident when making anomaly-related C. Model Complexity mistakes, especially false negatives. These observations suggest Figure 2(c) investigates the effect of model complexity by that calibration errors are not uniformly distributed across varying the number of trainable parameters in a multilayer classes. Instead, the primary calibration challenge arises perceptron. Performance metrics remain nearly identical across from overconfident mispredictions involving anomalous logs. all model sizes, with Acc and F1 consistently exceeding Consequently, improving confidence reliability for anomaly 0.999. Likewise, ECE and Brier score remain extremely small, predictions is likely to yield the greatest gains in overall suggesting excellent calibration according to conventional calibration quality, motivating the anomaly-focused route-wise metrics. Surprisingly, CoE increases monotonically as model calibration strategy adopted by LoRD. size grows, rising from 0.90 to nearly 0.99. This finding indicates that larger models become increasingly confident F. Characterizing Error Confidence The preceding analyses reveal a consistent pattern across on their mistakes despite exhibiting near-perfect conventional calibration scores. The result highlights a potential trade-off different architectures, hyperparameters, and training strategies. between model capacity and confidence reliability, where Although ECE and Brier score often indicate moderate calibraadditional representational power may amplify overconfidence tion performance, confidence on erroneous predictions (CoE) on rare erroneous predictions. Consequently, model scaling remains persistently high, frequently exceeding 0.9. This sugalone does not improve confidence quality and may even gests that models can appear well calibrated on average while still assigning near-maximal confidence to incorrect predictions. exacerbate error confidence. This discrepancy arises because conventional calibration metrics D. Solutions for Imbalances are dominated by correctly classified samples and therefore Figure 2(d) compares several commonly used strategies for provide limited insight into confidence behavior on rare addressing class imbalance. Traditional sampling-based meth- prediction errors. For log anomaly detection, such overconfident ods, including over-sampling, under-sampling, and balanced failures are particularly problematic because they may lead sampling, produce only modest improvements in calibration to missed anomalies or incorrect operational decisions. These behavior. Although these approaches slightly reduce CoE rela- observations consistently show that CoE remains a persistent tive to the baseline, error confidence remains consistently high. challenge regardless of model architecture, history window Among all methods, class-weighted loss achieves the largest re- length, adaptation rank, model complexity, or imbalanceduction in CoE, decreasing error confidence from 0.996 to 0.718 handling strategy. This finding suggests that confidence on while simultaneously reducing ECE and Brier score. Focal loss erroneous predictions is an important yet largely overlooked also improves calibration relative to the baseline but remains aspect of reliability in log anomaly detection. Consequently, less effective than class weighting. These results suggest that we introduce CoE as a complementary reliability metric and imbalance-aware optimization is more effective than data-level develop a reliability-aware calibration framework that explicitly resampling for reducing overconfidence. Nevertheless, even optimizes confidence quality on misclassified samples. the strongest baseline continues to exhibit substantial error V. L O RD S CALING confidence, indicating that class imbalance treatment alone In this section, we present Log Reconstruction and Discannot fully address the overconfidence problem. tance (LoRD), a lightweight calibration framework designed E. Class-wise Calibration to improve reliability in log anomaly detection. Building Figure 3 compares calibration behavior between normal and on the observation that confidence on erroneous predictions abnormal logs. A consistent class-wise disparity is observed remains persistently high across different detectors and training across all detectors. In terms of calibration metrics, abnormal settings, LoRD estimates prediction reliability from latent logs exhibit substantially larger ECE, NLL, and Brier scores representations and uses the resulting reliability scores to than normal logs, indicating that confidence estimates for recalibrate confidence. Specifically, LoRD combines route-wise anomaly predictions are considerably less reliable. The gap reconstruction modeling, asymmetric threshold and margin is particularly pronounced for LightLog, NeuralLog, and selection, and selective confidence adjustment to better align GPT2, where abnormal-class calibration errors exceed those prediction confidence with underlying prediction reliability. of the normal class by more than an order of magnitude. The Given an input log sequence xi , a trained anomaly detector confidence decomposition in Figure 3(d) further reveals the produces a hidden representation hi ∈ Rd , a predicted probsource of this disparity. While both normal and abnormal ability vector pi , and a predicted label ŷi ∈ {0, 1}. The true correctly classified samples (Nor. CoC and Abn. CoC) generally label is denoted as yi , where 0 indicates normal and 1 indicates

Fig. 4. Overview of LoRD with AE. Correctly classified validation samples are separated by prediction route to train route-specific AE. Reliability distances are then used to partition predictions into route-aware regions, which are subsequently mapped to confidence calibration policies.

anomalous. Our goal is to recalibrate the detector confidence so that high-risk predictions, especially anomalous samples incorrectly predicted as normal, receive lower confidence while preserving the detector’s original prediction labels.

architectures can be incorporated. We estimate the reliability distance of a sample using its reconstruction error: 2

di = hi − ĥi

, 2

(9)

where di measures the deviation of hi from the route-specific reliable manifold Mŷi . A small reliability distance indicates LoRD estimates prediction reliability through route-specific that the sample is well aligned with the reliable conditional conditional representation learning in the detector latent space. distribution of its prediction route, whereas a large distance Let hi denote the hidden representation of sample i, and suggests atypical latent behavior. Consequently, false negatives let ŷi , yi ∈ {0, 1} denote the predicted and ground-truth tend to deviate from the reliable normal manifold, while false labels, respectively. The key observation is that reliable normal positives tend to deviate from the reliable anomaly manifold. predictions and reliable anomaly predictions follow distinct The resulting reliability distance provides a unified measure latent distributions, of prediction reliability and serves as the basis for subsequent p(h | ŷ = 0, y = 0) ̸= p(h | ŷ = 1, y = 1). (6) confidence calibration. A. Representation Learning

Consequently, fitting a single reliability model to all reliable samples may obscure route-specific reliability structures and reduce the separability of prediction errors. To address this issue, LoRD learns route-specific reliability models that capture the conditional latent distributions of reliable predictions: Mr ∼ p(h | ŷ = r, y = r),

r ∈ {0, 1},

LoRD constructs route-specific calibration regions based on the reliability distance from the corresponding reliable manifold. To capture route-specific error patterns, LoRD learns (1) (2) two thresholds, τr < τr , for each prediction route and partitions samples into low-, mid-, and high-distance regions:

(7)

where Mr denotes the reliable latent manifold associated with prediction route r. For each route, LoRD learns a representation function fr (·) and a reconstruction function gr (·). Given a sample assigned to route ŷi , the corresponding route-specific model generates a reconstructed latent representation: ĥi = gŷi (fŷi (hi )) .

B. Route-Aware Reliability Partitioning

(8)

In this work, fr (·) and gr (·) are implemented as encoder– decoder networks, although alternative representation learning

Rlow = {di ≤ τr(1) }, r Rmid = {τr(1) < di ≤ τr(2) }, r

(10)

Rhigh = {di > τr(2) }. r (1)

(2)

Here, τr and τr are selected on the validation set according to the route-specific target recall Rr and flagged rate ρr . In practice, Rr and ρr are selected according to the operational risk tolerance of the application. Larger Rr increases the likelihood of identifying unreliable predictions, whereas smaller ρr limits unnecessary calibration of reliable

samples. The normal route (r = 0) prioritizes false-negative TABLE I ROUTE - REGION CALIBRATION POLICY MAP π. coverage, while the abnormal route (r = 1) prioritizes preserving reliable anomaly alarms. Rhigh Rmid Route Rlow r r r This partition reflects the route-dependent reliability structure. Normal (r = 0) HardAssign: p = 1 Identity SoftPull: p → 0.5 + ϵ In the normal route, samples with small reliability distance Abnormal (r = 1) SoftPull: p → 1 Identity HardAssign: p = 0.5 + ϵ are more likely to be reliable true negatives, whereas samples with large distance are more likely to be false negatives. In the abnormal route, samples with small reliability distance are Algorithm 1 LoRD Calibration more likely to be reliable true positives, whereas samples with Require: Detector fθ , validation set Dsel , test set Dtest , route targets {(R , ρr )}r∈{0,1} , calibration policy π large distance are more likely to be false positives. Samples in Ensure:r Calibrated probabilities pnew i the mid-distance region are treated as uncertain and are left 1: Obtain (ŷi , yi , pi , hi ) from fθ (Dsel ) for the reject option in the subsequent calibration policy. 2: ŷi , yi ∈ {0, 1}, where 0 is normal and 1 is abnormal C. Calibration Policies Based on the route-aware calibration map, LoRD assigns a calibration action to each route-region combination, as summarized in Table I. Predictions in the uncertain region are left unchanged through a reject option, while predictions in the reliable and high-risk regions undergo confidence adjustment according to their estimated log risk. The policy is designed to increase confidence for samples that are more likely to be safe logs and suppress confidence for samples that are more likely to be anomalous logs. Based on Table I, LoRD defines a route-region calibration policy π(r, z), where r denotes the prediction route and z denotes the reliability-distance region. Here, pi denotes the confidence assigned to the detector’s original predicted label. LoRD is label-preserving: it recalibrates the confidence score but does not recompute the predicted label after calibration. The Identity policy leaves pi unchanged. For the remaining regions, LoRD either directly assigns a target confidence through HardAssign or gradually moves pi toward a target confidence through SoftPull. The target confidence qr,z is determined by the selected route-region policy. Specifically, the normal low-distance region uses qr,z = 1, the normal highdistance region uses qr,z → 0.5 + ϵ, the abnormal low-distance region uses qr,z → 1, and the abnormal high-distance region uses qr,z = 0.5 + ϵ. For SoftPull, the adjustment strength is determined by the distance to the nearest calibration boundary:   ∆i αi = 1 − exp − , αi ∈ [0, 1], (11) s where ∆i denotes the boundary distance and s is a scale parameter determined by the corresponding margin. The recalibrated confidence is computed as pnew = (1 − αi )pi + αi qr,z , i

(12)

3: for r ∈ {0, 1} do 4: Define route-reliable set Tr = {i : ŷi = r, yi = r} 5: Define route-error set Er = {i : ŷi = r, yi ̸= r} 6: Train route autoencoder (fr , gr ) on {hi : i ∈ Tr } 7: Compute dri = ∥hi − gr (fr (hi ))∥22 for i ∈ Tr ∪ Er mid 8: Update Rlow , Rhigh under (Rr , ρr ) with Er r , Rr r 9: end for 10: for i ∈ Dtest do 11: Obtain (ŷi , pi , hi ) from fθ (xi ) 12: Set route r ← ŷi 13: Compute dri = ∥hi − gr (fr (hi ))∥22 14: Calibrate pi with π given dri 15: end for

for the normal and abnormal routes. First, the base detector is applied to a validation set to obtain predicted labels, confidence scores, and hidden representations. For each route r ∈ {0, 1}, correctly classified samples are treated as routereliable instances, while misclassified samples form the routeerror set. A route-specific autoencoder is then trained on the latent representations of reliable samples to learn the characteristic feature distribution of trustworthy predictions. Reconstruction distances are computed for both reliable and error samples, and these distances are used to construct low-, medium-, and high-reliability regions according to predefined route-specific targets. During inference, each test sample is assigned to its predicted route, and its reconstruction distance is evaluated using the corresponding route autoencoder. The resulting distance serves as a reliability indicator that is mapped through a calibration policy to adjust the detector’s original confidence score, producing calibrated probabilities that better reflect prediction reliability. VI. E XPERIMENTS A. Datasets

We evaluate our method on four large-scale supercomputing log datasets: BGL [31], Spirit [31], Liberty [31], and Thunwhere qr,z is specified by the selected calibration policy. Thus, derbird [31], covering anomaly ratios ranging from 0.49% to samples close to the boundary are only mildly adjusted, while 32.01%. Following prior work, we use 4.7M log messages from samples farther away are pulled more strongly toward the target new BGL, the first 5M log messages from Spirit and Liberty, and confidence. For HardAssign, LoRD directly sets pi = qr,z . 10M log messages from Thunderbird. To prevent information D. Algorithm leakage, each dataset is chronologically partitioned into nonWe summarize the calibration framework in Algorithm 1. overlapping training, detector-validation, selector-validation, LoRD performs route-specific confidence calibration by model- and test splits with a ratio of 7 : 0.5 : 0.5 : 2. The training split ing the latent representations of reliable predictions separately is used to optimize the base detector, the detector-validation

split is used for early stopping and checkpoint selection, and This demonstrates that the proposed route-wise reconstruction the selector-validation split is used for calibrator training mechanism selectively lowers confidence on risky predictions and threshold selection. When route-specific validation errors while retaining trustworthy anomaly detections. These results are unavailable in the selector-validation split, the detector- confirm that LoRD provides a task-driven calibration trade-off validation split is used as a fallback. The test split is reserved that is well suited for supervised log anomaly detection. exclusively for final evaluation. 2) Ablation study: Table III first evaluates the necessity of route-specific reliability modeling in LoRD. Replacing the B. Log Anomaly Detectors route-specific autoencoders with a single shared autoencoder We evaluate five widely used supervised log anomaly (SingleAE) consistently degrades abnormal calibration across detectors covering both conventional deep learning and lan- all datasets and detectors. For example, with LogRobust guage model-based approaches: TextCNN [12], which employs on Spirit, Abn. CoE increases from 0.508 to 0.516, while multi-scale one-dimensional convolutions to capture local with NeuralLog on Liberty it increases from 0.680 to 0.788. log patterns; LogRobust [13], which models log sequences These results suggest that predicted-normal and predictedusing an attention-enhanced bidirectional recurrent network; abnormal samples follow distinct reliability patterns in the LightLog [14], which combines compressed semantic log latent space. A shared reconstruction model must simulrepresentations with a lightweight temporal convolutional taneously characterize both distributions, which may blur architecture; NeuralLog [15], which leverages pretrained route-dependent reliability structures and reduce its ability language-model embeddings and a Transformer encoder to to distinguish reliable predictions from potential errors. In capture contextual dependencies; and GPT2 [17], which adapts contrast, route-specific modeling allows each autoencoder to a pretrained large language model for log anomaly detection learn a more homogeneous reliability manifold, resulting in through direct sequence modeling of log text. Unless otherwise improved separation between reliable and unreliable predictions. specified, all detectors are evaluated at the log-sequence level This observation is consistent with our hypothesis that normal using a sliding-window configuration with history length 10 and abnormal prediction routes exhibit different latent reliability and stride 1. characteristics and therefore benefit from separate modeling. We next evaluate the contributions of the two remaining C. Baseline Calibration Methods components of LoRD: the reject region mechanism and the We compare LoRD against five representative post-hoc cali- distance aware soft calibration strategy. Because confidence bration methods. Temperature Scaling (TempS) [27] calibrates calibration involves multiple objectives, including preserving confidence using pTS = softmax(zi /T ), where T is optimized confidence on correctly classified samples while suppressing i on a validation set. Logistic Scaling (LogS) [27] generalizes TS confidence on misclassified samples, we introduce two aggrethrough a learnable affine transformation, pLS gate measures to summarize overall calibration quality. Let i = softmax(w ⊙ zi + b). Beta Scaling (BetaS) [28] performs probability- x = (x1 , . . . , xm ) denote a set of calibration objectives. For level calibration using pBS = σ(a log pi + b log(1 − pi ) + c). each objective xj , we define a target value tj , where tj = 1 i Selective Scaling (SeleS) [29] applies separate temperatures for correct prediction confidence metrics and tj = 0.5 for error to selected high-risk and low-risk predictions based on output- confidence metrics. The objective deviation vector is defined as level signals such as confidence, entropy, or logit margin. z = (|x1 − t1 |, . . . , |xm − tm |) . We then define the aggregate Ensembling (Ens.) [29] averages predictions from multiple calibration distance as D = ∥z∥2 . To obtain a normalized PK − (k) 1 ∥2 independently trained detectors, pEns = K relative score, we further define C = ∥z∥∥z−z ,where z− i k=1 pi . All − 2 +∥z−z ∥2 calibration parameters are optimized using the validation set. denotes the worst achievable deviation vector. Smaller D and larger C indicate better overall calibration quality. D. Results Tables IV and V show that removing either the reject region 1) Performance: The main experiments show that LoRD or the soft calibration component consistently degrades overall consistently improves calibration reliability for anomalous calibration quality. For example, on Spirit with LogRobust, log detection across datasets and base detectors. In Table II, LoRD achieves the best performance with D = 0.405 and LoRD obtains the lowest CoE in nearly every dataset–model C = 0.783, outperforming both w/o Reject (D = 0.468, C = combination, indicating that it effectively suppresses overconfi- 0.754) and w/o Soft (D = 0.415, C = 0.776). Similar trends dent misclassifications without degrading the detector’s overall are observed on Liberty and with NeuralLog, indicating that the discrimination ability. Compared to standard post-hoc methods two components provide complementary benefits. These results such as Temperature Scaling, Logistic Scaling, Beta Scaling, demonstrate that route specific reliability modeling, the reject Selective Scaling, and Ensembling, LoRD yields substantially region mechanism, and distance aware soft calibration work lower error confidence while preserving reliable anomaly together to improve confidence reliability without sacrificing alarms. This behavior is pronounced for language-model-based anomaly detection performance. and high-capacity detectors, where LoRD reduces CoE from VII. C ONCLUSION values near one to values close to 0.5 in many settings. LoRD also maintains high confidence on correctly classified We propose LoRD, a lightweight post-hoc calibration anomalous samples, as reflected by its robustness in CoC values. framework for supervised log anomaly detection. By training

TABLE II C ALIBRATION PERFORMANCE MEASURED BY C O E ON ANOMALOUS LOG DETECTION ACROSS BENCHMARK DATASETS AND REPRESENTATIVE DETECTORS . R EPORTED VALUES DENOTE MEAN C O E WITH STANDARD DEVIATION OVER MULTIPLE RUNS , WHERE LOWER VALUES INDICATE BETTER PERFORMANCE . T HE BEST AND SECOND - BEST RESULTS IN EACH ROW ARE HIGHLIGHTED IN BOLD AND UNDERLINED ITALICS . Dataset

Model

Uncal

TempS

LogS

BetaS

SeleS

Ens.

LoRD

BGL

TextCNN LogRobust LightLog NeuralLog GPT2

0.977±0.003 0.987±0.009 0.829±0.022 0.990±0.002 0.998±0.000

0.976±0.004 0.973±0.016 0.983±0.007 0.975±0.007 0.992±0.000

0.988±0.002 0.988±0.004 0.967±0.009 0.974±0.007 0.993±0.000

0.989±0.003 0.988±0.003 0.979±0.004 0.988±0.001 0.991±0.000

0.968±0.005 0.907±0.098 0.831±0.015 0.847±0.016 1.000±0.000

0.966±0.005 0.748±0.163 0.998±0.000 0.985±0.001 0.998±0.000

0.540±0.006 0.577±0.092 0.702±0.036 0.509±0.002 0.566±0.002

Spirit

TextCNN LogRobust LightLog NeuralLog GPT2

0.890±0.055 0.957±0.007 0.959±0.014 0.996±0.003 0.936±0.000

0.862±0.051 0.936±0.005 0.983±0.007 0.975±0.007 1.000±0.000

0.776±0.091 0.929±0.009 0.967±0.009 0.974±0.007 0.982±0.009

0.833±0.056 0.950±0.011 0.979±0.004 0.988±0.001 0.942±0.000

0.858±0.039 0.930±0.021 0.929±0.010 0.972±0.009 0.998±0.001

0.926±0.009 0.951±0.010 0.948±0.029 0.991±0.006 0.936±0.001

0.556±0.022 0.506±0.004 0.514±0.014 0.508±0.007 0.500±0.000

Liberty

TextCNN LogRobust LightLog NeuralLog GPT2

0.979±0.002 0.937±0.050 0.959±0.015 0.995±0.005 1.000±0.000

0.972±0.003 0.930±0.052 0.960±0.020 0.994±0.006 0.993±0.003

0.881±0.025 0.944±0.020 0.925±0.044 0.982±0.007 0.994±0.000

0.933±0.013 0.979±0.007 0.979±0.007 0.995±0.003 0.988±0.001

0.949±0.016 0.958±0.036 0.861±0.022 0.995±0.004 1.000±0.000

0.967±0.002 0.952±0.038 0.916±0.030 0.995±0.006 1.000±0.000

0.506±0.002 0.502±0.002 0.505±0.001 0.683±0.018 0.590±0.004

Thunderbird

TextCNN LogRobust LightLog NeuralLog GPT2

0.852±0.020 0.926±0.033 0.656±0.016 0.852±0.087 1.000±0.000

0.884±0.007 0.929±0.048 0.973±0.036 0.947±0.061 1.000±0.000

0.910±0.027 0.902±0.093 0.975±0.038 0.951±0.062 1.000±0.000

0.999±0.000 0.995±0.004 0.999±0.001 0.991±0.005 0.994±0.010

0.708±0.052 0.898±0.036 0.905±0.063 0.902±0.087 1.000±0.000

0.737±0.013 0.706±0.076 0.723±0.048 0.881±0.060 1.000±0.000

0.561±0.040 0.619±0.123 0.500±0.000 0.516±0.021 0.534±0.008

TABLE III A BLATION STUDY OF L O RD ON BGL, S PIRIT, AND L IBERTY WITH L OG ROBUST AND N EURAL L OG AS BASE DETECTORS . A BN . C O C AND A BN . C O E DENOTE THE AVERAGE CONFIDENCE ON CORRECTLY AND INCORRECTLY CLASSIFIED ABNORMAL SAMPLES , RESPECTIVELY. Detector

Architecture

BGL

Spirit

Liberty

Abn. CoC↑

Abn. CoE↓

Abn. CoC↑

Abn. CoE↓

Abn. CoC↑

Abn. CoE↓

LogRobust

SingleAE Route-0-Only LoRD

0.562±0.066 0.993±0.005 0.990±0.003

0.582±0.038 0.542±0.002 0.541±0.002

0.556±0.041 0.999±0.001 0.999±0.000

0.516±0.011 0.583±0.018 0.508±0.003

0.740±0.049 0.999±0.001 0.986±0.008

0.506±0.002 0.617±0.050 0.502±0.001

NeuralLog

SingleAE Route-0-Only LoRD

0.549±0.052 0.998±0.001 0.998±0.001

0.610±0.023 0.506±0.002 0.506±0.002

0.647±0.111 0.998±0.001 0.999±0.001

0.543±0.059 0.575±0.059 0.511±0.008

0.783±0.033 1.000±0.000 1.000±0.000

0.788±0.200 0.716±0.044 0.680±0.047

TABLE IV A BLATION STUDY OF L O RD ON S PIRIT AND L IBERTY WITH L OG ROBUST AS THE BASE DETECTOR . C O C AND C O E DENOTE THE AVERAGE CONFIDENCE ON CORRECTLY AND INCORRECTLY CLASSIFIED SAMPLES , RESPECTIVELY. L OWER D AND HIGHER C INDICATE BETTER OVERALL CALIBRATION QUALITY. Spirit

Liberty

Method

Nor. CoC↑

Nor. CoE↓

Abn. CoC↑

Abn. CoE↓

D↓

C↑

Nor. CoC↑

Nor. CoE↓

Abn. CoC↑

Abn. CoE↓

D↓

C↑

w/o Reject w/o Soft LoRD

0.899 0.908 0.942

0.957 0.905 0.901

1.000 1.000 1.000

0.504 0.503 0.509

0.468 0.415 0.405

0.754 0.776 0.783

0.856 0.879 0.916

0.922 0.843 0.812

0.994 0.988 0.986

0.501 0.501 0.502

0.446 0.364 0.323

0.759 0.796 0.817

separate autoencoders for predicted-normal and predictedanomalous routes, LoRD captures route-specific reliability signals from detector latent representations and selectively recalibrates high-risk predictions while preserving the original detection labels. Experiments on multiple benchmark datasets and representative detectors show that LoRD effectively reduces confidence on misclassified anomalous samples, maintains trustworthy anomaly alarms, and outperforms standard scaling and ensemble-based calibration baselines. These results demonstrate the effectiveness of task-oriented, route-aware calibration for improving the reliability of real-world log anomaly detection

systems. An anonymous demo implementation of LoRD is available at: https://anonymous.4open.science/r/LoRD_code-E10B. VIII. L IMITATIONS LoRD has several limitations. First, its threshold and margin selection depend on route-specific validation errors, so the calibration boundary may be less stable when false negatives or false positives are rare. Second, LoRD is a task-oriented calibration method. It focuses on reducing the confidence of high-risk errors rather than improving all aggregate calibration metrics, which may lead to trade-offs in metrics such as ECE, NLL, or Brier score. Third, LoRD requires access to detector

TABLE V A BLATION STUDY OF L O RD ON ABNORMAL LOG CALIBRATION USING L OG ROBUST AND N EURAL L OG AS BASE DETECTORS . “ W / O R EJECT ” AND “ W / O S OFT ” REMOVE THE REJECT REGION AND S OFT P ULL CALIBRATION , RESPECTIVELY. L OWER D AND HIGHER C INDICATE BETTER OVERALL CALIBRATION . Spirit Method

CoC↑

CoE↓

Liberty D↓

C↑

CoC↑

CoE↓

D↓

C↑

0.994 0.988 0.986

0.501 0.501 0.502

0.446 0.364 0.323

0.759 0.796 0.817

1.000 1.000 1.000

0.572 0.602 0.680

0.450 0.395 0.354

0.757 0.790 0.814

LogRobust w/o Reject w/o Soft LoRD

1.000 1.000 1.000

0.504 0.503 0.509

0.468 0.415 0.405

w/o Reject w/o Soft LoRD

1.000 1.000 1.000

0.507 0.507 0.513

0.508 0.506 0.495

0.754 0.776 0.783 NeuralLog

hidden representations and is therefore more suitable for whitebox or gray-box detectors. Future work will extend LoRD to stronger distribution shifts and multi-class diagnosis. R EFERENCES [1] N. Han, S. Lu, D. Wang, M. Wang, X. Tan, and X. Wei, “Skdlog: self-knowledge distillation-based cnn for abnormal log detection,” in 2022 IEEE Smartworld, Ubiquitous Intelligence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles (SmartWorld/UIC/ScalCom/DigitalTwin/PriComp/Meta). IEEE, 2022, pp. 796– 805. [2] S. Lu, N. Han, M. Wang, X. Wei, Z. Lin, and D. Wang, “Ssdlog: a semi-supervised dual branch model for log anomaly detection,” World Wide Web, vol. 26, no. 5, pp. 3137–3153, 2023. [3] S. Lu, M. Wang, D. Wang, X. Wei, S. Xiao, Z. Wang, N. Han, and L. Wang, “Black-box attacks against log anomaly detection with adversarial examples,” Information Sciences, vol. 619, pp. 249–262, 2023. [4] S. Nedelkoski, J. Bogatinovski, A. Acker, J. Cardoso, and O. Kao, “Selfattentive classification-based anomaly detection in unstructured logs,” in 2020 IEEE international conference on data mining (ICDM). IEEE, 2020, pp. 1196–1201. [5] V.-H. Le and H. Zhang, “Log-based anomaly detection with deep learning: How far are we?” in Proceedings of the 44th international conference on software engineering, 2022, pp. 1356–1367. [6] M. Landauer, F. Skopik, and M. Wurzenberger, “A critical review of common log data sets used for evaluation of sequence-based anomaly detection techniques,” Proceedings of the ACM on Software Engineering, vol. 1, no. FSE, pp. 1354–1375, 2024. [7] J. Zhu, S. He, P. He, J. Liu, and M. R. Lyu, “Loghub: A large collection of system log datasets for ai-driven log analytics,” in 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 2023, pp. 355–366. [8] L. Wu, B. Lei, D. Xu, and D. Zhou, “Towards reliable rare category analysis on graphs via individual calibration,” in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 2629–2638. [9] M. Du, F. Li, G. Zheng, and V. Srikumar, “Deeplog: Anomaly detection and diagnosis from system logs through deep learning,” in Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, 2017, pp. 1285–1298. [10] W. Meng, Y. Liu, Y. Zhu, S. Zhang, D. Pei, Y. Liu, Y. Chen, R. Zhang, S. Tao, P. Sun et al., “Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs.” in Ijcai, vol. 19, no. 7, 2019, pp. 4739–4745. [11] L. Yang, J. Chen, Z. Wang, W. Wang, J. Jiang, X. Dong, and W. Zhang, “Plelog: Semi-supervised log-based anomaly detection via probabilistic label estimation,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 2021, pp. 230–231. [12] S. Lu, X. Wei, Y. Li, and L. Wang, “Detecting anomaly in big data system logs using convolutional neural network,” in 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big

0.736 0.741 0.747

Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), 2018, pp. 151–158. [13] X. Zhang, Y. Xu, Q. Lin, B. Qiao, H. Zhang, Y. Dang, C. Xie, X. Yang, Q. Cheng, Z. Li et al., “Robust log-based anomaly detection on unstable log data,” in Proceedings of the 2019 27th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, 2019, pp. 807–817. [14] Z. Wang, J. Tian, H. Fang, L. Chen, and J. Qin, “Lightlog: A lightweight temporal convolutional network for log anomaly detection on the edge,” Computer Networks, vol. 203, p. 108616, 2022. [15] V.-H. Le and H. Zhang, “Log-based anomaly detection without log parsing,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2021, pp. 492–504. [16] W. Guan, J. Cao, S. Qian, J. Gao, and C. Ouyang, “Logllm: Logbased anomaly detection using large language models,” arXiv preprint arXiv:2411.08561, 2024. [17] Y. F. Lim, J. Zhu, and G. Pang, “Adapting large language models for parameter-efficient log anomaly detection,” in Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 2025, pp. 325–337. [18] J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, P. Jung, R. Roscher et al., “A survey of uncertainty in deep neural networks,” Artificial intelligence review, vol. 56, no. Suppl 1, pp. 1513–1589, 2023. [19] R. Müller, S. Kornblith, and G. E. Hinton, “When does label smoothing help?” Advances in neural information processing systems, vol. 32, 2019. [20] G. Pereyra, G. Tucker, J. Chorowski, Ł. Kaiser, and G. Hinton, “Regularizing neural networks by penalizing confident output distributions,” arXiv preprint arXiv:1701.06548, 2017. [21] S. Thulasidasan, G. Chennupati, J. A. Bilmes, T. Bhattacharya, and S. Michalak, “On mixup training: Improved calibration and predictive uncertainty for deep neural networks,” Advances in neural information processing systems, vol. 32, 2019. [22] J. Mukhoti, V. Kulharia, A. Sanyal, S. Golodetz, P. Torr, and P. Dokania, “Calibrating deep neural networks using focal loss,” Advances in neural information processing systems, vol. 33, pp. 15 288–15 299, 2020. [23] Y. Cui, M. Jia, T.-Y. Lin, Y. Song, and S. Belongie, “Class-balanced loss based on effective number of samples,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9268– 9277. [24] Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in international conference on machine learning. PMLR, 2016, pp. 1050–1059. [25] B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” Advances in neural information processing systems, vol. 30, 2017. [26] A. G. Wilson and P. Izmailov, “Bayesian deep learning and a probabilistic perspective of generalization,” Advances in neural information processing systems, vol. 33, pp. 4697–4708, 2020. [27] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in International conference on machine learning. PMLR, 2017, pp. 1321–1330. [28] M. Kull, T. Silva Filho, and P. Flach, “Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers,” in Artificial intelligence and statistics. PMLR, 2017, pp. 623–631.

[29] D. Wang, B. Gong, and L. Wang, “On calibrating semantic segmentation models: Analyses and an algorithm,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23 652–23 662. [30] M. P. Naeini, G. Cooper, and M. Hauskrecht, “Obtaining well calibrated probabilities using bayesian binning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 29, no. 1, 2015. [31] A. Oliner and J. Stearley, “What supercomputers say: A study of five system logs,” in 37th annual IEEE/IFIP international conference on dependable systems and networks (DSN’07). IEEE, 2007, pp. 575–584.

Related documents

• Too Sure to Be Safe: Model Calibration for Reliable Log Anomaly Detection#1076938
• TRW: TRACE-RealWorld---An Auditable Consistency Contract for World Models as Materialized Views#411177
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems#346516
• NLLog: Lightweight, Explainable SOC Anomaly Detection via Log-to-Language Rewriting#259385
• Towards Deployable Anomaly Detection in industrial CPS/ICS: A Survey and Deployability Assessment#1098747