HSMLog: Small Language Model-Assisted Hardware Security Module Log Anomaly Detection with Behavioral Analysis

HSMLog: Small Language Model-Assisted Hardware Security Module Log Anomaly Detection with Behavioral Analysis Chia-Hsuan Wu∗ , Dar-Hsin Dustin Wu∗ , Rui Fang∗ , Yi-Ting Lee∗ , Chia-Chih Lin† , and Ming-Syan Chen∗ ∗ National Taiwan University, Taipei, Taiwan † National Taiwan University of Science and Technology, Taipei, Taiwan

arXiv:2608.29773v1 [cs.CR] 30 Aug 2026

Email: {chwu, dustinwu, rfang, ytlee}@arbor.ee.ntu.edu.tw, cclin@mail.ntust.edu.tw, mschen@ntu.edu.tw Abstract—Hardware Security Module (HSM) logs capture security-critical behavior, but anomalies emerge from relationships across event sequences, keys, object states, sessions, and temporal patterns rather than isolated events. Existing methods separate detection from HSM-specific evidence validation and reporting. In this paper, we present HSMLog, a two-stage framework for HSM log anomaly detection with retrieval-grounded behavioral analysis. In Stage 1, a small language model (SLM) identifies candidate alerts from sliding windows of structured HSM events and performs policy-guided assessment using HSMspecific operational rules. In Stage 2, retrieved policies and historical suspicious-key records strictly predating the alert window, together with candidate-related log context, support conservative candidate review and incident analysis. Evaluated on real industrial HSM background logs augmented with anomaly scenarios co-defined with industrial partners, HSMLog achieves 98.97% precision, 96.00% recall, 98.66% anomalous-event coverage, and a 97.46% F1 score, demonstrating effective anomaly alerting and incident triage in the studied setting. Index Terms—hardware security module, log anomaly detection, small language model, behavioral analysis

I. I NTRODUCTION Hardware Security Modules (HSMs) protect cryptographic keys and execute security-critical operations, including signing, encryption, decryption, key generation, object management, and authentication. Failures, misuse, and behavior that departs from deployment expectations can affect security, service availability, auditability, and the reliability of dependent applications. Detecting such conditions is therefore important for maintaining dependable HSM-backed services. Operational HSM logs differ from conventional system logs because each record describes a stateful cryptographic operation rather than only an event template or message. Its meaning depends on the target key or object, the session and role that issued the request, the object’s lifecycle state, the cryptographic mechanism, returned handles, and the result code. For example, an invalid-handle result may be expected after a stale reference but may indicate object exploration when repeated across unrelated objects. Likewise, a successful operation may still require investigation when it violates a deployment-specific policy. Consequently, behavior that appears benign at the event level may become suspicious only when interpreted as part of a structured operational progression.

Existing log anomaly detection methods have achieved promising results on public datasets [1]–[3], but their application to operational HSM environments remains challenging. Sequence-only detectors can overlook organization-specific policies, whereas policy-guided SLM assessment alone cannot capture every contextual workflow deviation. Either alone may miss repeated operations involving the same protected asset, requester, or invalid state transition, all central to HSM interpretation. Industrial settings rarely provide exhaustive event-level labels, with supervision often limited to annotated cases or incident spans. Given sensitive HSM logs, HSMLog uses a compact SLM over local windows, avoiding full-history inference; Stage 2 retrieves broader evidence only for detected candidates. This paper presents HSMLog, a two-stage framework for HSM log anomaly detection with retrieval-grounded behavioral analysis. Stage 1 uses an SLM to analyze structured event windows and assess organization-specific policies. Positive windows from contextual detection or policy-guided assessment are merged into candidate incidents. Stage 2 retrieves policies, suspicious-key records strictly predating the incident interval, and candidate-related log context, supporting conservative review and evidence-grounded behavioral analysis. The contributions of this paper are as follows: • We propose HSMLog, a two-stage framework that combines SLM-based contextual anomaly detection and policy-guided SLM assessment of organization-specific operational policies with retrieval-grounded candidate review and evidence-grounded behavioral analysis. • We develop a structured representation of HSM events and a case-aware sliding-window training protocol that enable contextual SLM detection from annotated operational scenarios. • We evaluate HSMLog on real operational logs from an industrial HSM environment using five practitionerinformed anomaly categories: Cryptographic Failure Probing, Object Handle Probing, Lifecycle Violation, Policy Violation, and Low-and-Slow Metadata Probing. We compare HSMLog with representative baselines and analyze case detection, anomalous-event coverage, component contributions, retrieval-supported candidate review, and observation-window sensitivity.

© 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

II. I NDUSTRIAL C ONTEXT AND P ROBLEM A. Hardware Security Modules and the PKCS#11 Interface In the studied environment, applications access HSM services through PKCS#11 (Cryptoki), a standard cryptographic token interface [4]. Unlike ordinary system logs, HSM audit records expose structured security-relevant entities: key or object handles and labels identify protected targets; session identifiers and roles indicate requesting authority; attributes and lifecycle operations describe object state; mechanisms specify cryptographic actions; and returned handles and result codes record outcomes. These fields must be interpreted jointly to determine whether an operation is consistent with an expected cryptographic workflow. B. Operational HSM Monitoring Challenges An HSM operation may be expected in one workflow but suspicious when its target, state, authority, or outcome conflicts with surrounding activity or policy. Anomaly evidence is therefore sparse, relational, and state-dependent, and not every contextual deviation can be specified in advance. This study uses anonymized industrial HSM audit data and anomaly scenarios co-defined with industrial partners. The partners helped identify relevant log fields and triage-oriented behavioral conditions, with sensitive content removed while preserving the event schema and behavioral structure. C. Problem Definition Let E = (e1 , e2 , . . . , eN ) denote a canonicalized HSM audit-event stream. HSMLog constructs a local observation window Wj = (ej , ej+1 , . . . , ej+w−1 ) and performs first-stage candidate detection as zj = dcontext (Wj ) ∨ dpolicy (Wj ),

zj ∈ {0, 1},

(1)

where dcontext denotes the SLM’s contextual anomaly decision and dpolicy denotes its policy-guided assessment against applicable organization-specific operational policies. Related positive windows are merged into candidate incidents to avoid duplicated alerts: C = Merge({Wj | zj = 1}) .

(2)

For each candidate ck ∈ C, Stage 2 retrieves operational policies Pk and suspicious-key records Hk− that strictly predate the candidate incident interval, and collects candidate-related log context Lk . The retrieved evidence and associated log context support conservative candidate review and, for retained candidates, retrieval-grounded behavioral analysis: âk = vreview (ck , Pk , Hk− , Lk ),

âk ∈ {0, 1}.

(3)

Here, âk = 1 indicates that candidate ck is retained for investigation; otherwise, it is not retained after evidence review. For each retained candidate, HSMLog generates a retrieval-grounded behavioral analysis rk from the retrieved evidence and associated log context. The objective is to

identify suspicious local behavior and retain candidates with sufficient operational support for engineer investigation. III. HSML OG A PPROACH A. Framework Overview Figure 1 presents the two-stage HSMLog workflow. After preprocessing constructs overlapping structured event windows, Stage 1 classifies each window as Anomaly or Normal and assesses it against applicable organization-specific policies from protected HSM knowledge, without disclosing confidential policy content or criteria. Positive windows from contextual detection or policy-guided assessment are merged into candidate incidents. This design combines learned workflow patterns with explicit operational constraints, covering both workflow deviations and policy-level misuse. Stage 2 applies only after a candidate incident is identified. It retrieves HSM policies, suspicious-key records strictly predating the incident interval, and candidate-related events for conservative review and retrieval-grounded behavioral analysis. It introduces no new detections, and this temporal restriction prevents the incident from supporting itself. B. Structured HSM Event Representation Raw HSM audit records contain interdependent fields whose independent treatment can obscure relationships among operations, keys, objects, sessions, and results. HSMLog converts each record into a compact, behavior-preserving representation for SLM inference. Figure 2 illustrates this preprocessing pipeline. The representation retains timestamps, session IDs, roles, operations, key handles and labels, mechanisms, attributes, search templates, returned handles, and result codes. List or dictionary attributes are normalized to text: [time] [Ses:session|role] operation(Hdl:handle|Lbl:label|Mech:mechanism) [Attr:attributes|Srch:template] → [Ret:handles|Result:result]

In this representation, Hdl and Lbl link operations on the same key or object; Ses and role preserve requesting context; and Mech, Attr, Ret, and Result preserve operation intent and outcome. This keeps security-relevant fields connected rather than treating them as independent tokens. The event stream enables SLM reasoning over object-state progression, cross-object access, repeated failures, and session-dependent usage beyond operation names. C. Sliding Windows and Case-Aware Training Protocol HSMLog uses overlapping local event windows over the canonical event stream, so related behavior can remain visible even when benign activity is interleaved, without assuming a single application session. For training, labels follow annotated scenario spans. A window is labeled Anomaly only when it fully contains an annotated case; partial overlaps remain Normal. This requires complete behavioral evidence but may place anomalous events in Normal-labeled windows near case boundaries, reflecting a conservative labeling trade-off.

Data Preprocessing

Stage 1: SLM-Based Contextual and Policy-Guided Anomaly Detection

1.1 Log Event Formatting

2. SLM-Based Contextual Detection

3. Policy-Guided SLM Assessment

LoRA-Adapted Instruction-Tuned SLM

Assess the window against applicable HSM policies

Raw HSM audit logs [14:03:21] [Ses:S-1842|User]

Input: 20-event window 𝑊𝑗

C_Sign(Hdl:0xA17|Lbl:jwt_signing_key

𝑊𝑗

|Mech:CKM_ECDSA_SHA256)

𝑒𝑗

𝑒𝑗+1

...

𝑒𝑗+19

Output: Anomaly / Normal

4. Candidate Incidents OR-merged contextual detection / policy assessment

OR

Attr:CKA_SIGN,CKA_PRIVATE → CKR_OK

Sent to Stage 2 for review and analysis

Output: Anomaly / Normal

1.2 Sliding Window Construction Event sequence 𝑒1

𝑒2

𝑒3

...

𝑊𝑗

𝑒𝑗

𝑒𝑗+1

...

𝑒𝑗+19

𝑊𝑗+1

𝑒𝑗+1

𝑒𝑗+2

...

𝑒𝑗+20

Stage 2: Retrieval-Grounded Candidate Review and Behavioral Analysis 5. Retrieval Evidence Base Historical Suspicious-Key Records

HSM Policies

6. SLM-Assisted Retrieval-Grounded Candidate Review

Associated Log Context

Review candidate incidents using supporting evidence

Candidate-related log context

1.3 Case-Aware Training Labeling Anomaly case (complete) ...

Filtered Out

Retained Incident

N

N

A

A

A

A

N

N ...

N

N

A

A

A

A

N

N

7. Retrieval-Grounded Behavioral Analysis

Anomaly

Summarize suspicious behavior and supporting evidence

Full Containment N

N

A

A

A

Partial Overlap

A

Output: Behavioral Analysis Report

Normal

Fig. 1. Overview of HSMLog. Stage 1 performs SLM-based contextual and policy-guided anomaly detection over structured HSM windows. Stage 2 uses retrieved policies, prior suspicious-key records, and candidate-related log context for conservative review and retrieval-grounded behavioral analysis.

1. Raw HSM Audit Records 2026-04-18 13:52:57 | Ses:0x3B65F | CKU_USER C_SignInit | keyA | CKM_ECDSA → CKR_OK 2026-04-18 13:53:12 | Ses:0x3B44A | CKU_USER C_FindObjectsInit | label=keyB → CKR_OK

2. Behavior-Preserving Canonicalization Selected canonical fields session/role

time

object/search

operation

result

mechanism

𝑒1

𝑒2

[13:52:57 | Ses:0x3B65F | CKU_USER] C_SignInit(keyA | CKM_ECDSA) → CKR_OK

[13:53:12 | Ses:0x3B44A | CKU_USER] C_FindObjectsInit(label=keyB) → CKR_OK

3. Overlapping 20-Event Windows for SLM Event sequence

𝑒1

𝑒2

𝑒3

𝑒4

...

𝑒18

𝑒19

𝑊𝑗

𝑒𝑗

𝑒𝑗+1

𝑒𝑗+2

...

𝑒𝑗+19

𝑊𝑗+1

𝑒𝑗+1

𝑒𝑗+2

𝑒𝑗+3

...

𝑒𝑗+20

𝑒20

...

Fig. 2. Behavior-preserving preprocessing for HSM logs. Raw records are canonicalized into compact events and organized into overlapping event windows for SLM classification. Only selected log fields are shown for clarity.

D. Stage 1: SLM-Based Contextual Detection HSMLog uses an instruction-tuned small language model to classify each canonicalized HSM event window as Anomaly

or Normal using an HSM-specific instruction. The instruction directs the model to assess sequence coherence and contextual consistency across events, sessions, keys, objects, and outcomes, including object-state progression, access relationships, session-dependent behavior, mechanism usage, and result-code patterns. This enables HSMLog to identify behavior that appears benign in isolation but becomes suspicious through repeated failures, cross-object access, lifecycle inconsistencies, or other HSM-specific context. E. Stage 1: Policy-Guided SLM Assessment Some behaviors depend on organization-specific policies beyond contextual detection. For each window, HSMLog supplies the SLM with applicable protected rules and log context for policy-guided assessment, then combines this decision with contextual evidence. The same rules support Stage 2 review and explanation, not new detection; confidential details remain undisclosed. To avoid duplicate alerts, overlapping positive windows are merged into a single candidate incident. F. Stage 2: Retrieval-Grounded Candidate Analysis Stage 2 provides post-detection analysis using RetrievalAugmented Generation (RAG) [5]. For each candidate incident, HSMLog uses its implicated keys, operations, and time interval to retrieve relevant policies and strictly earlier suspicious-key records, while collecting candidate-related

TABLE I T RAINING AND FINAL - TEST ANOMALY COMPOSITION . Category

Train

LSLM = −

1 X ωt log pθ (yt | y<t , Wj ). |T |

(4)

t∈T

Test

Cases

Events

Cases

Events

Cryptographic Failure Probing Object Handle Probing Lifecycle Violation Policy Violation Low-and-Slow Metadata Probing

45 45 45 0 45

732 736 577 0 692

21 19 20 21 19

344 313 258 363 294

Total

180

2,737

100

1,572

events from the merged incident span. These inputs support conservative review: Stage 2 filters only weak, unsupported candidates, introduces no new detections, and uses no groundtruth labels. For each retained incident, HSMLog organizes the same evidence into a retrieval-grounded behavioral analysis that identifies affected keys or objects, summarizes suspicious progression, states supporting evidence, and highlights investigationrelevant context for post-detection explanation and triage. IV. E VALUATION S ETUP A. Dataset Construction The evaluation combines real operational HSM logs with anomaly scenarios co-defined with industrial partners. Training contains 27,319 events: 24,582 normal-background events from 352 sessions and 2,737 anomalous events from 180 labeled cases. The contextual SLM is trained on four categories; Policy Violation is assessed only through policy-guided assessment at inference. The final test stream contains 16,573 events: 15,001 normalbackground events and 1,572 scenario-instantiated anomalous events across 100 near-balanced cases in five categories. Normal backgrounds are session-disjoint, and training and finaltest anomaly instances are independently instantiated with disjoint sessions, keys, handles, and event windows. Table I summarizes the anomaly composition. B. Implementation Details Stage 1 uses a predefined observation window of w = 20, fixed before final-test evaluation to preserve broader context for operations separated or interleaved by routine activity. The contextual detector uses google/gemma-4-E4B-it [6] with LoRA [7] applied to linear modules with rank r = 16, scaling factor 32, and dropout 0.05. Fine-tuning processes 78.85 million tokens across 54,600 window instances (two training passes over 27,300 windows), using a learning rate of 2×10−5 and response-only supervision with loss applied only to classification completions. This focuses optimization on the classification decision rather than reproducing the input prompt. To reflect the higher operational cost of missed anomalies, Anomaly completion tokens receive increased loss weight. Let T denote the response-token positions, where ωt = 50 for anomaly-response tokens and ωt = 1 otherwise. The training objective is

C. Anomaly Scenario Definitions The practitioner-informed scenarios comprise controlled HSM monitoring conditions motivated by cryptographichardware attack research, stealth reconnaissance strategies, and the stateful object-management semantics of PKCS#11 [4], [8]–[11]. The scenarios also include a deployment-specific policy condition grounded in PKCS#11 operational constraints [12]. They are not confirmed production incident traces or a fixed taxonomy from prior HSM audit-log studies. • Cryptographic Failure Probing: Repeated cryptographic requests produce concentrated related failures for a common target, reflecting error-conditioned probing behavior observed in attacks against cryptographic hardware [8]. • Object Handle Probing: PKCS#11 object-search operations return handles for matching token or session objects [4]. Repeated searches, accesses, or invalid-handle attempts involving workflow-unjustified objects indicate possible reconnaissance of token-resident assets [10]. • Lifecycle Violation: A client reuses an object handle after a destructive operation invalidates its associated object. This scenario tests whether monitoring preserves objectstate progression across events [4]. • Policy Violation: A syntactically valid operation conflicts with a deployment-specific policy. It represents a policygoverned violation rather than a generic public attack family [12]. • Low-and-Slow Metadata Probing: A client intermittently queries unrelated keys or objects during routine activity, adapting low-rate stealth reconnaissance strategies to HSM metadata access [9]. Together, these scenarios test whether HSMLog preserves the key/object, session, state, and result relationships needed to recognize HSM-specific behavior. D. Baselines We compare HSMLog with DeepLog [1], LogRobust [3], PLELog [13], LogBERT [2], NeuralLog [14], LogLLM [15], and MIDLog [16]. These methods span sequential prediction, robust representations, parsing-free semantic encoding, weak supervision, self-supervision, and language-model-based analysis. They do not use HSMLog’s policy-guided assessment, retrieval review, or behavioral-analysis components. This comparison evaluates HSMLog against generic methods in the studied HSM setting. Policy Violation is an unseen policy-driven test category. Baselines assess it with native scoring or classification because they do not receive HSMLog’s organization-specific policies. All baselines were trained and evaluated using the same session-disjoint data splits and case-aware incident protocol as HSMLog. Their configurations and decision thresholds were fixed before final-test evaluation.

E. Case-Aware Alert-Incident Evaluation All methods use a common case-aware alert-incident protocol with one-to-one incident-case matching, preventing multiple credits for repeated alerts. Cryptographic Failure Probing, Object Handle Probing, and Policy Violation require at least 50% case-event coverage. Lifecycle Violation requires 75% coverage for its statetransition pattern. Low-and-Slow Metadata Probing requires at least five overlapping events, or all events in shorter cases. Case detection rate (CDR) is the fraction of anomaly cases matched by an alert incident. Event coverage (EC) is the fraction of anomalous events covered by matched incidents. Precision, recall, and F1 are computed over alert incidents. V. R ESULTS AND D ISCUSSION A. Comparison with Representative Baselines Table II reports category-wise CDR and EC together with overall EC, precision, recall, and F1. HSMLog achieves the highest recall and F1 score: 96.00% recall, 98.66% EC, 98.97% precision, and a 97.46% F1 score. PLELog is the strongest baseline in terms of F1 at 92.71%; HSMLog improves F1 by 4.75 points and recall by 7 points. EC alone is insufficient: NeuralLog attains 98.85% EC, 0.19 points above HSMLog, but only 89.00% recall and 91.75% F1. HSMLog detects seven more cases while retaining 98.66% EC and improving precision from 94.68% to 98.97%. B. Contribution of Contextual Detection, Policy-Guided Assessment, and Retrieval-Grounded Candidate Review Table III(a) isolates the contributions of contextual SLM detection, policy-guided assessment, and retrieval-based candidate review. Contextual SLM alone achieves 78.00% recall, 68.70% EC, and 81.25% F1, showing contextual patterns from structured windows but incomplete coverage. Policy-guided assessment alone achieves 100.00% precision but 21.00% recall, showing policy knowledge cannot cover all evaluated patterns. Combining both raises recall to 96.00%, EC to 98.66%, and F1 to 96.97%, improving recall by 18 points over contextual SLM alone. Retrieval review raises precision from 97.96% to 98.97% and F1 from 96.97% to 97.46%, with unchanged recall and EC. The 1.01-point precision gain and 0.49-point F1 gain reflect Stage 2’s bounded triage role, not a second detector. C. Observation-Window Trade-off Table III(b) reports a post-hoc sensitivity analysis of Stage 1 under alternative observation-window sizes. Although w = 10 yields higher recall and F1, it provides lower anomalous-event coverage than w = 20. We fixed w = 20 before final-test evaluation to favor broader preservation of candidate-related evidence, which is important for subsequent retrieval-grounded review and behavioral analysis, rather than maximizing F1 alone. The results therefore illustrate a tradeoff between incident detection performance and anomalousevent coverage. Beyond w = 20, recall drops as unrelated background dilutes localized evidence.

D. Role of Retrieval-Grounded Behavioral Analysis For each retained incident, HSMLog organizes retrieved policy evidence, pre-incident suspicious-key context, and the associated candidate event sequence into a retrieval-grounded behavioral analysis. The analysis identifies the affected key or object, summarizes suspicious progression, states supporting evidence, and highlights investigation-relevant context. The analysis follows Stage 1 detection and Stage 2 review rather than serving as a free-form additional detector. We evaluate Stage 2 as a bounded candidate-review mechanism, not as a substitute for engineer judgment. The generated analysis supports post-detection explanation and triage. VI. R ELATED W ORK DeepLog [1] models normal sequences; LogBERT [2] learns contextual representations through masked language modeling; LogRobust [3] targets unstable or noisy logs; and NeuralLog [14] uses neural representations to reduce explicittemplate dependence. PLELog [13] and MIDLog [16] address limited or inexact supervision, while LogLLM [15] explores language-modelbased log analysis. HSMLog applies an SLM directly to structured HSM windows rather than only for post-hoc explanation, combines learned context with policy-guided assessment, and uses retrieval-grounded review and analysis to connect detection with operational investigation. VII. L ESSONS L EARNED AND L IMITATIONS Three practitioner lessons emerge. First, HSM monitoring should preserve key/object, session, state, and timing context. Second, learned contextual detection benefits from deployment-specific policy knowledge. Third, post-detection review should remain conservative and evidence-grounded. These principles may extend to other stateful, policy-governed systems, including key-management and access-control systems and regulated workflows, with domain-specific log representations and policies. This study has limitations. Normal background data derive from real industrial operation, whereas anomaly cases are practitioner-informed scenarios co-defined with industrial partners and embedded in the evaluation stream; they are meaningful HSM anomaly conditions, not confirmed production incidents. Evaluation is limited to one HSM environment and structured log schema; some policy conditions are only functionally described to protect confidential operational knowledge. Stage 2’s usefulness to HSM engineers was not directly evaluated; future work should study this with practitioners and additional HSM vendors, workloads, and policies. VIII. C ONCLUSION This paper presented HSMLog, a two-stage framework that combines SLM-based contextual detection and policy-guided assessment over structured HSM windows with retrievalgrounded candidate review and behavioral analysis. On real

TABLE II C ASE DETECTION AND ANOMALOUS - EVENT COVERAGE BY CATEGORY, WITH OVERALL INCIDENT- MATCHING PERFORMANCE . Cryptographic Failure Probing

Object Handle Probing

CDR

EC

CDR

EC

CDR

EC

CDR

EC

CDR

EC

EC

Prec.

Rec.

F1

DeepLog LogRobust PLELog LogBERT NeuralLog LogLLM MIDLog

90.48 80.95 95.24 95.24 95.24 90.48 85.71

92.15 83.14 95.93 95.93 97.97 90.99 86.63

89.47 100.00 94.74 94.74 100.00 89.47 89.47

94.89 100.00 95.21 95.21 100.00 90.10 90.10

50.00 65.00 60.00 60.00 60.00 60.00 55.00

79.46 93.80 60.47 60.47 100.00 60.47 54.26

90.48 90.48 100.00 100.00 100.00 85.71 85.71

100.00 100.00 100.00 98.35 96.97 87.33 87.33

89.47 94.74 94.74 94.74 89.47 84.21 84.21

96.60 96.94 94.22 96.60 100.00 85.71 85.71

93.26 94.72 90.59 90.65 98.85 83.97 82.00

93.18 90.53 96.74 93.68 94.68 94.25 100.00

82.00 86.00 89.00 89.00 89.00 82.00 80.00

87.23 88.21 92.71 91.28 91.75 87.70 88.89

HSMLog

100.00

95.06

94.74

100.00

90.00

98.45

100.00

100.00

94.74

100.00

98.66

98.97

96.00

97.46

Method

Lifecycle Violation

Policy Violation

Low-and-Slow Metadata Probing

Overall

CDR = case detection rate; EC = anomalous-event coverage. All values are percentages. Boldface indicates the best value in each column. TABLE III C OMPONENT ABLATION AND OBSERVATION - WINDOW SENSITIVITY OF HSML OG . (a) Component ablation

(b) Stage 1 window sensitivity

Configuration

EC

Prec.

Rec.

F1

w

EC

Prec.

Rec.

F1

Contextual SLM only Policy-guided assessment only Stage 1: Contextual + Policy-guided Full HSMLog: Stage 1 + Retrieval Review

68.70 23.09 98.66 98.66

84.78 100.00 97.96 98.97

78.00 21.00 96.00 96.00

81.25 34.71 96.97 97.46

10 20 25 30 50

96.12 98.66 78.63 78.18 86.07

98.99 97.96 100.00 100.00 100.00

98.00 96.00 70.00 69.00 64.00

98.49 96.97 82.35 81.66 78.05

industrial HSM background logs augmented with anomaly scenarios co-defined with industrial partners, HSMLog achieves 98.97% precision, 96.00% recall, 98.66% anomalous-event coverage, and a 97.46% F1 score, demonstrating effective alerting and incident triage in the studied setting. ACKNOWLEDGMENT This work was conducted with technical guidance and support from eMemory Technology Inc. and PUFsecurity. R EFERENCES [1] M. Du, F. Li, G. Zheng, and V. Srikumar, “DeepLog: Anomaly detection and diagnosis from system logs through deep learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2017, pp. 1285–1298. [2] H. Guo, S. Yuan, and X. Wu, “LogBERT: Log anomaly detection via BERT,” in International Joint Conference on Neural Networks. IEEE, 2021, pp. 1–8. [3] X. Zhang, Y. Xu, Q. Lin, B. Qiao, H. Zhang, Y. Dang, C. Xie, X. Yang, Q. Cheng, Z. Li, J. Chen, X. He, R. Yao, J.-G. Lou, M. Chintalapati, F. Shen, and D. Zhang, “Robust log-based anomaly detection on unstable log data,” in Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 2019, pp. 807–817. [4] OASIS, “PKCS #11 Specification Version 3.1,” OASIS Open, OASIS Standard, Jul. 2023. [5] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 9459–9474. [6] Google DeepMind, “Gemma 4 Model Card,” https://ai.google.dev/ gemma/docs/core/model card 4, 2026, accessed: 2026-07-03.

[7] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [8] R. Bardou, R. Focardi, Y. Kawamoto, L. Simionato, G. Steel, and J.K. Tsay, “Efficient padding oracle attacks on cryptographic hardware,” in Advances in Cryptology – CRYPTO 2012, ser. Lecture Notes in Computer Science, vol. 7417. Springer, 2012, pp. 608–625. [9] M. Dabbagh, A. J. Ghandour, K. Fawaz, W. E. Hajj, and H. Hajj, “Slow port scanning detection,” in 2011 7th International Conference on Information Assurance and Security. IEEE, 2011, pp. 228–233. [10] M. Bortolozzo, M. Centenaro, R. Focardi, and G. Steel, “Attacking and fixing PKCS#11 security tokens,” in Proceedings of the 17th ACM Conference on Computer and Communications Security. ACM, 2010, pp. 260–269. [11] S. Delaune, S. Kremer, and G. Steel, “Formal security analysis of PKCS#11 and proprietary extensions,” Journal of Computer Security, vol. 18, no. 6, pp. 1211–1245, 2010. [12] M. Centenaro, R. Focardi, and F. L. Luccio, “Type-based analysis of PKCS#11 key management,” in Principles of Security and Trust, ser. Lecture Notes in Computer Science, vol. 7215. Springer, 2012, pp. 349–368. [13] L. Yang, J. Chen, Z. Wang, W. Wang, J. Jiang, X. Dong, and W. Zhang, “Semi-supervised log-based anomaly detection via probabilistic label estimation,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 2021, pp. 1448–1460. [14] V.-H. Le and H. Zhang, “Log-based anomaly detection without log parsing,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2021, pp. 492–504. [15] W. Guan, J. Cao, S. Qian, J. Gao, and C. Ouyang, “LogLLM: Log-based anomaly detection using large language models,” arXiv preprint arXiv:2411.08561, 2024. [Online]. Available: https://arxiv.org/ abs/2411.08561 [16] M. He, T. Jia, C. Duan, H. Cai, Y. Li, and G. Huang, “Weaklysupervised log-based anomaly detection with inexact labels via multiinstance learning,” in Proceedings of the 47th IEEE/ACM International Conference on Software Engineering, 2025, pp. 2918–2930.

Related documents

• Bulkhead: Automated Semantic Detection and Remediation of Container Escape Vulnerabilities#366193
• Decoupled Descent: Exact Test Error Tracking Via Approximate Message Passing#146951
• SmartHomeSecure: Automated Detection and Repair of Smart Home Configuration Errors Using Large Language Models#349701
• Cryptographic Message Syntax (CMS) Key Package Receipt and Error Content Types#399067
• Signed Syslog Messages#501228