From IOCs to Regex: Automating CTI Operationalization for SOC with LLMs Pei-Yu Tseng∗ , Lan Zhang† , ZihDwo Yeh∗ , Xiaoyan Sun‡ , Xushu Dai∗ , Peng Liu∗ ∗ The Pennsylvania State University, PA, USA † Northern Arizona University, AZ, USA ‡ Worcester Polytechnic Institute, MA, USA ∗ [email protected], [email protected], [email protected], [email protected]
arXiv:2604.12228v1 [cs.CR] 14 Apr 2026
† [email protected] ‡ [email protected]
Abstract—Cyber Threat Intelligence (CTI) reports provide Indicators of Compromise (IOCs) that are essential for security operations. A common way to operationalize IOCs is through regular expressions (regexes), which enable precise matching of attack traces across heterogeneous log data. Regexes support a wide range of tasks, including digital forensics, log parsing, and the construction of rules in Security Information and Event Management (SIEM) systems. However, regex construction remains largely manual: analysts must extract IOCs from CTI reports and then craft syntactically valid and semantically precise patterns. This process is slow, error-prone, and increasingly unsustainable given the rising volume of CTI reports. While recent approaches have applied Large Language Models (LLMs) to IOC extraction, they primarily focus on producing raw artifacts—leaving IOCs as plain strings that still require manual transformation into regexes before they can be operationalized. This gap limits their practical utility, as plain strings cannot account for variations in system contexts, log formats, or attacker behaviors, all of which regexes are designed to capture. To address these challenges, we propose a fully automated LLM-based regex generation system, IOCRegex-gen, that generates corresponding regexes from IOCs. Our system introduces two key innovations: (i) a group-aware mechanism that automatically distinguishes which IOC segments should be treated as capture groups versus non-capture groups, and (ii) an iterative reasoning and multi-stage validation pipeline that ensures regexes are both syntactically valid and semantically correct. Our experimental validation using over 3,000 real CTI reports and over 2,400 ground truth strings collected from the MITRE ATT&CK Evaluation framework demonstrates that the regexes generated by our system achieve an average hit rate of 99.1% across the datasets, with an average false-positive rate of only 0.8%, confirming its practical utility for large-scale CTI processing and automated regex generation.
1. Introduction In modern Security Operations Centers (SOCs), analysts rely on a wide variety of data sources collected on a daily basis, including CTI reports, Windows event logs, endpoint logs, and Linux journals. To utilize these data sources in a
scalable way, they must be parsed automatically to find the specific information items that analysts are searching for. To serve the purpose of IOC hunting, CTI reports are a valuable resource, as they describe attacker behaviors and provide IOCs such as file paths, registry keys, and command-line arguments. However, there is a fundamental gap between how attacker behaviors are described in CTI reports and how they are recorded in specific system logs. For example, a CTI report may describe an attacker creating a scheduled task with the command schtasks.exe \create \tn ..., but in system logs, the same activity may appear with different path delimiters, execution parameters, or capitalization, making exact string matching infeasible. Without bridging this gap, even if IOCs are successfully extracted from CTI reports, threat intelligence remains difficult to operationalize in practice. The de facto strategy to bridge this gap is the use of regexes. In practice, human analysts manually translate IOCs extracted from CTI reports into regexes, which can then be executed by computers to search massive log data. Regexes thus provide the medium that operationalizes human-readable intelligence into actionable detection, supporting tasks such as digital forensics, log parsing, and the construction of SIEM rules. When constructing regexes for IOCs, security analysts typically employ a multi-step workflow. They first review CTI reports from cybersecurity companies, individual experts, or knowledge bases such as MITRE ATT&CK, extract the relevant IOCs, and then manually translate them into regexes. Although this manual process provides a very important enabler for many advanced SOC operations, it faces critical limitations that have grown more severe as the frequency and scale of global attacks increase. [1]. In practice, the current multi-step workflows face several critical limitations. First, the end-to-end process of reading CTI reports, authoring regexes, and subsequently testing and tuning them can take hours to days, creating bottlenecks that delay responses to emerging threats. Second, regex authoring expertise is unevenly distributed: novice analysts often resort to senior colleagues for help, straining already limited security team resources. Third, the repetitive and detail-oriented nature of regex construction contributes to analyst fatigue, leading to errors such as syntax mistakes,
missing capture groups, or insufficient generalization. These limitations introduce mounting time and labor pressures on SOC operations. Recent research alleviates these limitations by automating part of the multi-step workflow. One line of work focuses on IOC extraction from CTI reports. For instance, [2] utilized a pre-trained named-entity recognition (NER) model to extract IOCs from text and constructed a graph to represent inter-IOC relationships, while [3] applied a pretrained model to identify candidate threat actions for timely cyber defense. More recently, [4] and [5] employed LLMs to extract IOCs from CTI texts. While these extraction approaches are valuable, their functionalities stop at producing raw IOCs and do not address the subsequent—and equally critical—step of translating these IOCs into deployable regexes. Consequently, analysts must still manually perform this regex construction, leaving the core operational bottleneck unresolved. Beyond IOC extraction, several recent studies [5], [6], [7] have attempted to take one further step by leveraging LLMs to generate SIEM detection rules from CTI data automatically. However, in these works, the proportion of generated rules that incorporate regexes remains unrealistically small. For instance, [6] utilized the in-context learning capabilities of LLMs to perform Tactics, Techniques, and Procedures (TTPs) classification and generate detection rules informed by external threat intelligence databases. While their results include 135 automatically generated rules, the vast majority correspond to surface-level indicators such as IP addresses and domain names, with only 7 rules incorporating any form of regex-based pattern matching, and these regex components are very simple in terms of structure and significantly simpler than the regexes in realworld SIEM rules, reflecting very limited exploration of regex synthesis in current approaches. Another line of work has explored regex generation in different contexts, though not specifically for security IOCs. These approaches can be broadly categorized into two groups. Traditional Natural Language Processing (NLP) methods, such as [8] and [9], improved syntactic validity through sequence-to-sequence models but were designed for controlled natural language tasks with well-formed input descriptions. More recently, LLM-based methods such as [10] and [11] have extended regex generation capabilities through improved reasoning and multi-step inference. However, these approaches remain centered on general natural language-to-regex translation and lack mechanisms to handle the unique challenges of security contexts: IOC variants exhibit adversarial variations intentionally designed to evade detection; there is a requirement for balancing precision with generalization to catch attack variations; and there is a requirement to maintain operationally meaningful capture groups for downstream SIEM integration. Furthermore, unlike general regex generation tasks, where training examples are abundant, real-world IOCs are scarce, sensitive, and constantly evolving, making it infeasible to collect large sets of positive and negative examples for data-driven learning approaches such as supervised training or model fine-tuning.
Consequently, a clear gap exists between the existing regex generation approaches and the requirements for dealing with the adversarial and resource-constrained nature of security operations. In this work, we aim to take the critical remaining step of translating raw IOCs into deployable regexes. As implied by the above discussion, this step involves significant challenges. (C1) Regex construction requires determining which substrings of an IOC should serve as capture groups for extracting key threat indicators. Misplacing capture groups not only weakens detection accuracy but also disrupts downstream integration with SIEM rules and forensic analysis tools, where extracted values are used for correlation and investigation. (C2) Regex generation with LLMs is prone to hallucinations and errors—models may produce syntactically invalid patterns or semantically misaligned expressions that fail to match the intended IOC behavior. Without robust mechanisms for validation and correction, such outputs cannot be safely deployed in production environments. (C3) Regex generation must balance precision and generality: overly narrow patterns miss true attack actions, while overly broad ones—such as those dominated by wildcards or optional groups—yield little practical detection value. To address these challenges, we propose IOCRegexgen, a novel LLM-based system that automates the translation from extracted IOCs to deployable regexes—the critical missing step in the multi-step workflow. Our system accepts IOCs extracted by any existing method as input and addresses the three challenges through two core components. The first component employs a knowledge-enhanced framework integrated with a graph database to identify which substrings of each IOC should be designated as capture groups (addressing C1). By retrieving relevant threat intelligence context and analyzing structural patterns across similar IOCs, this component ensures that capture groups align with operational requirements for SIEM integration and forensic analysis. The second component implements a reasoning-based generation process with iterative validation to ensure both syntactic correctness and semantic accuracy (addressing C2 and C3). Through multi-stage validation checks and refinement loops, the system mitigates LLM hallucinations, prevents the generation of syntactically invalid patterns, and maintains an appropriate balance between precision and generality by rejecting overly broad or overly narrow regex candidates. To validate our system’s effectiveness, we conduct experiments using the MITRE ATT&CK Evaluation framework [12], which provides independently collected artifacts from actual security incidents. Specifically, the independently collected artifacts are reported by cybersecurity companies during these MITRE ATT&CK evaluation exercises. We let these artifacts serve as ground-truth. We then generated regexes from over 3,000 real-world CTI reports using our system and evaluated their performance against these ground-truth strings. Our experimental results demonstrate that IOCRegex-gen achieves strong detection performance: the generated regexes attain an average hit rate of 99.1% on ground-truth strings while maintaining a low false-positive
rate of 0.8%. Furthermore, quantitative analysis shows that the generated regexes achieve an average grading score above 3, indicating substantial structural complexity beyond simple literal matches. The average similarity score between generated regexes and their source IOCs is around 0.4, suggesting that the patterns retain essential IOC features while introducing sufficient variability for broader applicability. Ablation experiments further confirm the importance of each system component. Compared to directly prompting an LLM to generate regexes without our specialized components, IOCRegex-gen achieves over 30% improvement in both hit rate and false-positive rate, demonstrating that our design effectively addresses the challenges inherent in CTIto-regex translation. In summary, we have made the following contributions: • We propose IOCRegex-gen, an LLM-based system that automates the translation from extracted IOCs to deployable regexes, addressing the critical missing step in the CTI operationalization workflow and bridging the gap between threat intelligence extraction and actionable security detection. • We develop novel methods for automated capture group identification through knowledge-enhanced context retrieval and graph-based pattern analysis, coupled with a reasoning-driven generation workflow that performs iterative validation to ensure syntactic correctness, semantic accuracy, and appropriate generalization. • We conduct a comprehensive evaluation using over 3,000 real-world CTI reports and ground-truth strings from the MITRE ATT&CK Evaluation, demonstrating that our system achieves a 99.1% hit rate with a 0.8% false-positive rate and outperforms direct LLM baselines by over 30% in both metrics. Open Source: The source code for our implementation is available at https://anonymous.4open.science/r/IOCRegex-g en-8FFE/README.md
2. Background 2.1. Cyber Threat Intelligence CTI reports serve as an essential medium for cybersecurity analysts to exchange threat intelligence, providing detailed insights into attackers’ methods, workflows, and technical procedures. For example, [13] documents the background, targets, and tactics of the hacker group TeamTNT. While some CTI reports are paid resources, many are freely accessible and contributed by both organizations and individuals. Major cybersecurity vendors such as Trend Micro and FireEye routinely publish high-quality CTI reports [13], [14], [15], whereas individual researchers and SOC analysts often share technical analyses on blogs or personal platforms [16], [17]. In addition, centralized repositories like MITRE ATT&CK [18] and vendor-specific platforms such as Trend Micro’s Threat Encyclopedia [19] aggregate CTI content for broader accessibility. 2.1.1. Regex-Applicable IOCs. The IOCs embedded within CTI reports serve as clear indicators to determine whether
After being loaded, the backdoor writes to the HKCU\Software\Microsoft\Windows\CurrentVersion\Run registry key ... ..... The attackers copied ccf32.exe to \\ <remote_host>\C$\Users\Public\folder, along with a bat file (e.g.\\ <remote_ host>\C$\Users\Public\11.bat), then executed the bat daily, using schtasks.exe: schtasks /create /s <remote_host> /u “<username>” /p “<password>” /ru “SYSTEM” /tn one /sc DAILY /tr “c:\ users\public\11.bat” /F ...... the mechanisms used are the Run registry key and the Startup folder, .....
Figure 1: Example paragraphs from a CTI report
a system has been breached. However, not all types of IOCs are suitable for regex generation. According to the Pyramid of Pain [20], the low-value indicators—such as hash values, IP addresses, domain names, and network artifacts—are the ones most easily changed by adversaries. These indicators are highly volatile, as attackers can regenerate file hashes, rotate or proxy IP addresses, and frequently modify domain names or network configurations. They are also instance-specific and lack consistent syntactic structure, which makes it difficult to generalize them into meaningful regexes. For example, a hash value such as d41d8cd98f00b204e9800998ecf8427e is unique to a single file and provides no reusable pattern that a regex could capture. In practice, these indicators are more effectively handled through exact-match lookups, threat intelligence feeds, or reputation-based systems rather than pattern matching. In contrast, file paths, registry keys, and commandline arguments represent the most critical and regexapplicable types of IOCs, since they reliably reveal persistent threats and inherently contain both fixed and variable components—such as stable system directories combined with attacker-controlled filenames or parameters. These structural characteristics make them difficult to operationalize through exact matching but highly suitable for regexbased representation, which is essential for enabling their use in large-scale searching and SOC tasks. In Figure 1, the text illustrates how the attacker leverages the Chinoxy backdoor to achieve persistence on the victim machine and to schedule regular execution of a data-collection utility. First, the attacker drops the backdoor binary into the victim’s userprofile directory and writes its full path to the registry key HKCU\Software\Microsoft\Windows\CurrentVe rsion\Run. Thereafter, whenever the user logs into Windows, the backdoor automatically launches, ensuring it remains resident. In addition, the attacker uses Windows’ built-in scheduling tool schtasks.exe to create a daily task on the remote host that executes the data-collection program. In this text, the registry key entries, the path to the malicious executable, and the attacker’s command line argument invocations constitute critical indicators of attack. When these indicators appear on a system, they may signify that the system is under attack or has already been compromised. Security analysts closely scrutinize CTI reports to identify IOCs, as these indicators are essential for establishing SIEM rules or guiding digital forensics.
2.1.2. IOC Extraction. Extracting IOCs from CTI reports is an essential step in transforming unstructured textual threat descriptions into structured and actionable data. Traditionally, analysts manually review CTI reports to locate IOCs. However, as the scale and complexity of CTI reports continue to grow, manual extraction has become increasingly time-consuming and prone to inconsistency, motivating the development of automated approaches. Recent studies have explored diverse NLP-based approaches to automate IOC extraction. Some works employ pre-trained models for named entity and relation extraction to identify threat entities and contextual relationships in CTI narratives [2], [3], [21]. Others LLMs to directly interpret CTI text, extracting not only IOCs but also associated TTPs, and constructing structured threat knowledge graphs [22], [23], [24], [25], [26]. For instance, [22] annotates entities and relationships to enhance downstream model training, and [23] builds CTI-oriented knowledge graphs to represent threat behaviors and mitigate hallucinations. Despite such progress, existing work treats IOC extraction as an intermediate to serve non-regex-construction purposes, such as knowledge graph construction or TTPs identification. While these approaches greatly advance the structuring and interpretation of CTI, the operational use of extracted IOCs—for example, transforming them into deployable regex for digital forensics—remains less explored in the current literature. 2.2.InRegular Expressions SOC operations, regex in is Security commonlyOperations used because, despite the usefulness of IOCs from CTI reports, they typically cannot be directly matched with system logs for two key reasons. First, each machine’s system context is unique. IOCs may reflect the specific context in which the CTI report was written, so even if the attacker uses the same command line argument, the actual implementation may vary slightly across different environments. For example, when an attacker uses the command line argument in Figure 1 to create a scheduled task, they might change the path of the invoked program depending on the machine’s system context, or adjust the execution time based on the machine’s timezone. Furthermore, relying on exact matches against raw IOCs is impractical; even slight differences in capitalization, path delimiters, or whitespace will often cause missed detections. If pattern matching relied solely on literal strings, variations like temp1.exe, temp2.exe, tmpA34.dll would evade detection due to their minor naming differences. Using regex, however, allows these payloads to be matched collectively by identifying shared elements like filename prefixes and extensions. Therefore, after obtaining IOCs, security analysts construct regexes to efficiently identify string variations that are similar to, but not exactly the same as, the original IOCs—making largescale IOC searching in massive log datasets both feasible and reliable. In practice, the widespread adoption of regex across commercial SIEM platforms further underscores its operational importance in SOC workflows. Many SIEM systems natively employ regex, enabling analysts to flexibly filter complex log patterns and reduce irrelevant results. Splunk provides “rex” and “regex” com-
mands for direct pattern searches [27], Elastic SIEM supports regex in its query language for large-scale indexed data [28], and IBM QRadar integrates regex into its property extraction and correlation rules [29]. Moreover, we analyzed 1,735 public Splunk detection rules [30] and found that over 64% contained regex patterns related to file paths, registry keys, or command-line arguments, indicating that these elements frequently serve as matching targets in practical SOC detection logic.
3. Motivation 3.1. SOC Regular Expression Generation Workflow In real-world SOCs, security analysts typically manually generate regexes. In particular, the workflow is as follows: upon receipt of a CTI report (which can range from a few pages to several thousand words), the analyst first reviews the document to determine which techniques the adversary employs, and then extracts a raw list of IOCs from the narrative descriptions of those techniques. Next, the analyst manually deduplicates overlapping entries, filters out low-value IOCs, and retains the high-value IOCs mentioned in Section 2.1.1. Building on the remaining IOC list, the next step for a SOC analyst is to decompose each IOC into its mutable and immutable elements, denoted in regex as non-capture groups and capture groups, respectively. Non-capture groups represent segments of an IOC that can change between machine’s system context or due to attacker variation, such as randomized file names, GUIDs, or ephemeral registry key values, while capture groups correspond to constant, unchanging components like system-built directory paths or built-in command names, which threat actors habitually use to make their behavior appear normal [31]. For example, given the file path IOC C:\Users\Public\11.bat, an analyst might define the path prefix Users\Public as a capture group, and the payload name 11.bat as a noncapturing group. Security analysts identify these parts by combining their institutional knowledge of OS conventions and common adversary tradecraft with external documentation, such as Microsoft’s Sysinternals guides or CTI vendor IOC repositories, to determine which elements truly require flexibility. Once grouping is delineated, they construct an initial regex that combines literal matches, quantifiers, and alternations (e.g. .*\Users\Public\.*), then iterate on it using regex testing tools like regex101 [32] to confirm it’s syntactically valid, not overly permissive, and precise enough to catch IOC variants without false positives. Only once the regex pattern passes the validation is it deployed into the SOC’s SIEM system.
3.2. LLM-based system to Automate the Workflow By leveraging an LLM-based system, the regex generation workflow could be automated through the inherent capabilities provided by LLMs. However, utilizing LLMs to complete this task presents several challenges that must
CTI reports
Manual Workflow
CTI reports
Directly using LLMs CTI reports
Our AI agent
...the backdoor writes to the HKCU\Software\Microsoft\Windows\ CurrentVersion\Run registry key.... copied ccf32.exe to the folder, along with a bat file (e.g.C:\Users\Public\11.bat), then executed daily, using: schtasks /create /s /u “” /p “” /ru “SYSTEM” /tn one /sc DAILY /tr “c:\users\public\11.bat” /F
schtasks, HKCU...
Generate and Refine
Regex (?i).*schtasks.... (?i)HKCU\\Software\\.......
...the backdoor writes to the HKCU\Software\Microsoft\Windows\ CurrentVersion\Run registry key.... copied ccf32.exe to the folder, along with a bat file (e.g.C:\Users\Public\11.bat), then executed daily, using: schtasks /create /s /u “” /p “” /ru “SYSTEM” /tn one /sc DAILY /tr “c:\users\public\11.bat” /F ...the backdoor writes to the HKCU\Software\Microsoft\Windows\ CurrentVersion\Run registry key.... copied ccf32.exe to the folder, along with a bat file (e.g.C:\Users\Public\11.bat), then executed daily, using: schtasks /create /s /u “” /p “” /ru “SYSTEM” /tn one /sc DAILY /tr “c:\users\public\11.bat” /F
IOCs
Read
C:\Users\Public\11.bat
Overlooking
folder
Misidentified
(?i)C:\\Users\\Public\\11.bat
Cannot distinguish captue group
(?i)C:\\\\\\\\\\11.bat
Incorrect regex
Regex
AI agent
(?i).*schtasks.... (?i)HKCU\\Software\\.......
Figure 2: Motivating Example be overcome. As discussed in Section 2.1.2, prior research has primarily focused on automatic IOC extraction. In this section, we focus on the remaining challenges specific to the regex generation workflow. First, current LLMs lack sufficient knowledge to distinguish which parts of an IOC belong to the capture group versus the non-capture group (C1). Unlike experienced analysts, who rely on domain knowledge of operating system conventions, LLMs often struggle with this distinction. For instance, using the example mentioned in Section 3.1, where the IOC C:\Users\Public\11.bat could also appear as 12.bat, a human analyst would recognize that the stable path prefix Users\Public should be treated as a capture group, while the filename is attacker-controlled and better represented as a non-capture group. LLMs, by contrast, lack the contextual understanding to make this separation reliably. To address this limitation, our system employs a knowledge-enhanced context retrieval method, which will be shortly presented in Section 4.1. This method enriches the capture group finding process with structured system knowledge that reflects the intrinsic organization of operating environments. Specifically, it incorporates canonical information about file path hierarchies, registry key structures, and command-line utility patterns—elements that remain consistent across machines and provide stable references for determining which IOC components should be treated as capture groups. Second, LLMs frequently fail to produce syntactically valid and complete regexes, particularly when multiple capture groups or nested patterns are involved (C2). Although LLMs have demonstrated strong capabilities in logical tasks such as mathematical reasoning and code generation, their application to regex synthesis remains largely unexplored. Our observations show that LLMs frequently produce invalid or incomplete regexes when tasked with longer or more intricate patterns, and they often omit required capture groups specified in the prompt. Previous studies have primarily applied reasoning-based methods to domains such as mathematical problem solving and code generation, where model feedback and external tools are used to iteratively
refine outputs toward the desired solution [33], [34]. In our system, we implement a dedicated reasoning workflow for regex generation: using this method, the LLM incrementally constructs a correct and valid regex step by step, ensuring that the required capture groups are preserved. Third, even when LLMs generate syntactically valid regexes, these patterns can often be overly general and thus useless in practice (C3). For instance, the model may produce regexes dominated by wildcards or optional groups, such as (...)?, resulting in patterns that technically compile but match almost everything, thereby providing no discriminative power. To mitigate this issue, our system builds upon the reasoning-based method described above to guide the model away from excessively permissive constructions. In addition, we introduce a grading system that evaluates candidate regexes against a set of criteria and discards those that fail to provide meaningful specificity, thereby ensuring that only regexes with practical detection value are retained.
4. Approach We propose IOCRegex-gen, a system that transforms raw IOCs from CTI reports into syntactically valid and semantically precise regexes. Since our goal is to automate the translation of IOCs into regexes, the system focuses exclusively on regex-applicable IOC mentioned in Section 2.1.1, and accepts IOC outputs from any existing extraction framework. Figure 3 provides an overview of the overall system. Generally, IOCRegex-gen can be divided into two phases. In the Capture Group Finding phase, which addresses the first challenge (C1) discussed in Section 3.2, ① IOCRegex-gen retrieves from a pre-established graph database the relationships between substrings in the potential IOC strings, ② applies algorithms to determine which substrings are capture groups, and then, using a designed filtering mechanism, ③ removes false positives identified in the IOC extraction. The subsequent Regex Generation phase targets the second challenge (C2) and third challenge (C3) identified in Section 3.2. During this phase, ④ IOCRegexgen uses a tool to evaluate the LLM’s proposed regex against the target IOC. The tool’s output, which details any discrepancies between the suggested regex and the IOC, is then used to craft the next prompt to guide the LLM toward an improved regex. This iterative reasoning process continues until the tool reports no discrepancies. At the same time, the reasoning process also ensures that the regex is not overgeneralized and does not include any non-capturing groups. Finally, ⑤ IOCRegex-gen leverages the above-mentioned reasoning process to repeatedly generate multiple regexes for the same IOC. ⑥ A scoring mechanism is then applied to evaluate these candidates and select the highest-scoring regex. The selected pattern becomes the final regex for that IOC.
4.1. Capture Group Finding Building on the definitions introduced in Section 3, our regex generation pipeline must include a dedicated grouping
IOCRegex-gen 1. Capture group Finding CTI report
2. Regex Generation Reasoning
IOC strings HKCR\CLSID\... %APPDATA%\.. EhStorShell.dll loader ...
HKCR\CLSID\... HKCR\CLSID\... HKCR\CLSID\... EhStorShell.dll EhStorShell.dll EhStorShell.dll ntshrui.dll ntshrui.dll ......%APPDATA%\.. ...
Filtering Regex debugging
③
Final Regex Non-Capture Group Validation
④
HKCR\CLSID\... EhStorShell.dll ntshrui.tlb ...
Capture Group Finding
IOCs ② ①
Retrieve
Neo4j
Windows docu.
HKCR\CLSID\... APPDATA.. .dll ...
⑤
Regular Expression Grading
⑥
OverGeneralization Check
Figure 3: Overview of IOCRegex-gen step that isolates invariant IOC fragments from those likely to vary across environments or attacker campaigns. In this phase, IOCRegex-gen retrieves data from an external graph database and uses our proprietary algorithm to differentiate between capture groups and non-capture groups within each string extracted during IOCs Extraction. Additionally, when encountering any string that does not contain a capture group, it is considered a false positive from the previous stage and is subsequently removed, and only the strings that remain after filtering are considered genuine IOCs.
4.1.1. Graph Database. As mentioned in Section 3.1, most attackers tend to use the operating system’s built-in command line arguments, registry keys, or directories to conceal their malicious activities. These built-in components are treated as capture groups. Therefore, we have collected all native file paths, registry keys, and command line arguments for Windows 8, 10, and 11, as well as Windows Server 2012, 2016, 2019, and 2022. To facilitate efficient retrieval of data for capture group finding in section 4.1.3, this data is stored in the graph database as a tree structure. We create the tree structure for file paths, registry keys, and command line arguments separately using the following methods. 1) We split file paths or registry keys using slashes as delimiters to obtain substrings. In the tree structure, each substring is treated as an individual entity, and each entity is connected by a link established according to the hierarchical order, effectively associating them with one another. For example, as shown in Figure 4, under the Users root directory, there may be several subdirectories named after individual users. Here, the Users directory serves as the parent node, its subdirectories are child nodes, and each of those child nodes may in turn contain further subdirectories as their own child nodes. 2) When constructing the tree structure for the command line arguments, we first determine which element represents the command and which ones are the subordinate parameters. For example, curl is a tool used for sending and receiving various network requests. It has many parameters, such as --get, --request, --data, and others. We then establish parent and child relationships from curl to its parameters in the tree structure, reflecting their hierarchical dependency.
curl
Users
User -get
-request
Public
-data AppData
Local
Command Line argument
Download
Roaming
File Path
Figure 4: The tree structure of command line argument and file path 4.1.2. Preprocessing. Typically, CTI reports are written by humans without a standardized format, so it is necessary to standardize the extracted strings. For instance, a registry key’s root (e.g., HKEY_CURRENT_USER) might appear as HKCU or simply as REGISTRY. Similarly, when representing a user’s personal folder, the path might be expressed as C:\Users\<username>, <username>, or even user. During preprocessing, We first classify the string obtained from IOCs Extraction based on its structure to determine whether it is a file path, registry key, or command line argument, and then handle several common cases: 1) Environment Variable Expansion: For file paths that may be expressed using environment variables, such as %USERPROFILE%, we restore them to their original form. 2) Username Normalization: Various representations of usernames are standardized by replacing them with a uniform token (e.g., user). 3) Registry Key Standardization: For all registry keys, the root keys are standardized by using their abbreviated forms. 4) Command Line Argument Normalization: For command line arguments, the commands (e.g., cmd.exe, curl.exe, or wmic.exe) are normalized by removing file extensions. 4.1.3. Capture Group Finding. After preprocessing the potential IOC strings, file paths, and registry keys are split into individual substrings using slashes, while command line arguments are split based on whitespace, semicolons, and other delimiters to extract substrings. Since the structure of command line arguments differs from that of file paths and registry keys, we employ two distinct algorithms to identify capture groups within these strings. In Algorithm 1, the input S is a list of substrings that
are ordered according to their hierarchical position within the path, and D represents the graph database. Additionally, V is defined as the set of substrings in list S that appear as entities in the graph database. Let t be an entity in the graph database; if t is adjacent to s in the graph, we denote this relationship as sAt. For a given list S, we start by verifying which substrings in the list appear in the graph database (line 1). This initial step filters out substrings that are clearly not native to the operating system, for example, file or folder names arbitrarily assigned by an attacker. Next, we take the first remaining substring and add it to the current sequence C. We then check if the next substring is adjacent to the previous one in the D. If they are adjacent, we add the substring to C and continue checking the following substrings in the same manner (lines 3-8). If a substring is not adjacent to the previous one, we record the length of the current sequence C and restart the process with the non-adjacent substring as the new starting point. This iterative process continues until we identify the longest sequence, L, of adjacent substrings (lines 6-11). We retain only the longest sequence as the capture group because we’ve observed that some directories native to the operating system often recur across different file paths. Additionally, this approach helps prevent capturing directory names that may have been arbitrarily defined by attackers. For example, consider the file path Algorithm 1: Capture group finding for file path and registry key Input: S = ⟨s1 , s2 , . . . , sn ⟩, si ∈ Σ Input: D, the graph database Data: V ← ⟨s ∈ S | s ∈ D⟩ Data: s At ⇐⇒ t is the adjacent node of s in D Output: L = ⟨ℓ1 , ℓ2 , . . . , ℓk ⟩, k = max{ |C| : C ⊆ V, ∀ j, V j AV j+1 } V ← ⟨s ∈ S | s ∈ D⟩; max ← 0, L ← ⟨ ⟩; 3 for i ← 1 to |V | do 4 C ← ⟨Vi ⟩; 5 j ← i; 6 while j < |V | and V j AV j+1 do 7 C ← C ◦V j+1 ; 8 j ← j+1; 1
category. If so, it is added to the current sequence C (lines 5-10). The algorithm then checks whether the subsequent substring exists in D and belongs to the parameter category. If it does and is adjacent to the previous substring in the graph, it is added to C; otherwise, it is discarded (lines 11-12). This process is repeated until the next substring identified as a command is encountered (lines 5-14). The final output is a collection of sequences Ω, where these sequences represent the capture groups for the string related to the command line argument. For Algorithm 2: Capture group finding for command line argument Input: S = ⟨s1 , s2 , . . . , sn ⟩, si ∈ Σ Input: D, the graph database Data: G(s, label) returns true if s is marked as the given label (command or parameter) in D. Data: s At ⇐⇒ t is the adjacent node of s in D Output: Ω = ⟨ω1 , ω2 , . . . , ωk ⟩, where each ω is an ordered sequence of substrings starting with a command followed by its parameters V ← ⟨s ∈ S | s ∈ D⟩; Ω ← ⟨ ⟩; 3 C ← ⟨⟩ 4 c ← undefined 5 for i ← 1 to |V | do 6 if G(Vi , “command”) is true then 7 if C ̸= ⟨ ⟩ then 8 Ω ← Ω ◦ ⟨C⟩; 1 2
9 10 11 12
C ← ⟨Vi ⟩; c ← Vi ; else if G(si , “parameter”) is true and c AVi then C ← C ◦ ⟨Vi ⟩;
14
if C ̸= ⟨ ⟩ then Ω ← Ω ◦ ⟨C⟩;
15
return Ω;
13
2
9 10 11
if |C| > max then max ← |C|, L ← C; return L;
C:\Users\Public\11.bat in Figure 1. The algorithm first checks whether the directory Users exists as a node in the graph database. If it does, it then verifies that Public is a child node of Users in the tree structure. Next, it checks whether 11 is a child node under Public. If it is not, then the capture group for that file path is Users\Public. In Algorithm 2, the input consists of a list S of substrings along with the graph database D. However, the output is a set of sequences Ω because a single command line argument may contain multiple main commands. Given a list S containing multiple ordered substrings, the algorithm starts with the first substring and verifies whether it exists in D and belongs to the command
instance, using the command line arguments schtasks /create /s <remote_host>/u "<username>" /p "<password>" /ru "SYSTEM" /tn one /sc DAILY /tr "c:\users\public\11.bat" /F shown in Figure 1 as an example, the algorithm first identifies the command, schtasks. It then uses the tree structure in the graph database to determine which substrings are child nodes of that command (e.g., /create, /s, /u, etc.), and simultaneously flags any substrings not found in the tree, such as <remote_host>, <username>. Finally, it treats those substrings absent from the tree as non-capture groups, and all remaining substrings as capture groups. 4.1.4. False Positive Filtering. After the capture group finding phase, IOCRegex-gen processes each string from IOCs Extraction by labeling every component as either keep or discard. Components marked as keep are considered to be part of the string’s capture group, while those labeled as discard are treated as part of the non-capture group. Finally, if a string does not contain any capture group components, IOCRegex-gen treats it as a false positive from the IOCs Extraction phase and discards it.
4.2. Regex Generation Prior studies have shown that regex generation can be automated through rule-based or NLP-based methods. However, these approaches are ill-suited for IOC-related regex generation. Methods such as [35] require large numbers of representative positive examples, which are difficult to obtain for IOCs. Rule-based approaches also struggle with the diverse formats and structures of IOCs. Moreover, since each IOC may involve different capture groups, existing methods lack the flexibility to generate accurate regexes across varying IOC types. 4.2.1. Reasoning-based Regex Generation. With the advent of LLMs, it initially appeared that these models could address these challenges by generating regex patterns for IOCs. Compared to using a smaller deep learning model to generate regexes [36], the massive parameter scales of LLMs provide significant improvements in textual and logical processing. However, we observed that regardless of the LLM (e.g., GPT-4o, DeepSeek-V3, or LLaMA3) used, none can consistently generate correct regex patterns for IOCs. In most cases, the regex output still contains noncapture groups, resulting in non-compliant regex patterns. Furthermore, we observed that even newer reasoning models, such as GPT-O4 or Deepseek-R1, which seem promising in solving these issues, struggle when faced with lengthy IOCs like complex command line arguments. In such cases, these reasoning models prove ineffective. To address these limitations, we propose a novel workflow that incrementally guides the LLM to generate compliant and accurate regex patterns. Remarkably, this workflow does not require the use of advanced reasoning models; using a standard LLM (e.g., GPT-4o) yields results that surpass those of current reasoning models. As shown in Figure 5, the process works as follows: Capture group & IOC
LLM
If loop > 10 times, restart
Initial regex
If all match (over-generalized), restart
Is match with IOC?
NO
Valied regex
Is non-capture group in regex?
Testing with 10 random strings
Regex debugger
LLM
YES
Word finding
Candidate regex
Figure 5: Workflow of Reasoning-based Regex Generation Initial Regex Generation: The IOC, along with the capture group information obtained from the previous phase, is fed into an LLM to generate an initial regex. Debugging. The generated initial regex is tested to see if it can successfully match the original IOC. If it fails to match, a regex debugger is used to identify the exact step
where the failure occurs. The debugger provides details on which parts of the regex can match and which cannot. This information is then used in a prompt to the LLM, which iteratively refines the regex until a valid one is obtained. Non-Capture Group Validation. Once the regex has passed the debugging checks to confirm it still matches the original IOC, we run a second loop to ensure that it contains no capture groups. Within this loop, if any non-capture group components are found, relevant prompts indicating which non-capture groups are still present in the regex are fed back into the LLM along with the current regex, triggering further iterative refinement until a valid regex with no noncapture group elements is produced. If either the debugging loop or the non-capture group refinement loop exceeds ten iterations, it returns to the very beginning of the workflow to generate a new initial regex. Over-Generalization Check. Even after obtaining a valid regex that matches the original IOC, it is sometimes observed that the LLM generates a regex composed entirely of wildcards(e.g., ‘.*’ or ‘+w’), leading to over-generalization. To address this, in addition to the two loops mentioned above, we set up a third loop. In this loop, ten strings are randomly generated and tested to see whether the candidate regex matches all of them. If the regex matches every generated string, it is considered overly generic; it then restarts the workflow from the very beginning to create a new initial regex.
4.3. Regular Expression Grading In addition to the reasoning-based regex generation workflow described in Section 4.2.1, we also employ a separate mechanism to ensure that the generated regexes genuinely include the intended capture groups and do not match any strings belonging to non-capture groups. Since the same IOC can be matched by multiple different regexes. Based on our observations, LLMs (e.g., GPT-4o) generate, on average, about five distinct regex variations. Exploiting this characteristic, we use the workflow from Section 4.2.1 to generate five candidate regexes for each IOC, then score each regex using the formula below and retain only the highest-scoring regex as the final regex. In this scoring formula, 1) A regex receives points for each component that belongs to a capture group. 2) It incurs a penalty for each non-capture group included. The formula for scoring is as follows: Score = α ncg − β nwc
(1)
ncg represents the number of components that belong to capture groups, and nwc represents the number of noncapture groups. Furthermore, the LLM may insert substrings into the regex that do not belong to any capture group; such substrings are also included in nwc . The reward coefficient α and penalty coefficient β determine the relative importance of capturing essential components versus avoiding substrings that should not be captured. We enforce a more stringent policy regarding the inclusion of non-capture
Type
File Path Registry CLI Args Other
Count
12,195
2,302
10,286
38,820
TABLE 1: Distribution of extracted IOC types groups. Therefore, we set α = 1 and β = 1 to balance the scoring. It’s worth noting that if a capture group appears inside an optional group, such as ‘(...)?’, we do not include it in the scoring.
5. Evaluation Our evaluation addresses five research questions. (RQ1) Can IOCRegex-gen generate regexes in a scalable way? (RQ2) What are the false positive rates of the generated regexes? (RQ3) Are the generated regexes too general, causing unintended matches? (RQ4) Are the regexes general enough to detect attack pattern variations? (RQ5) What is the individual contribution of each system component?
5.1. Experimental Setup In this experiment, we used GPT-4o, LLaMA3, and DeepSeek-V3 to extract the IOCs and employed GPT-4o to generate the regexes, while the graph database was implemented using Neo4j. Additionally, all datasets and intermediate process data were stored in JSON format. The entire system is built with over 4000 lines of Python code. 5.1.1. IOC Dataset for Regex Generation. We collected 3,156 CTI reports from the references listed under the MITRE ATT&CK framework [18]. Each ATT&CK technique web-page contains multiple external references, which we systematically crawled and parsed to obtain a diverse corpus of CTI reports. To ensure a consistent and comprehensive dataset of IOC inputs for evaluation, we adopted an extraction process inspired by recent LLM-based CTI parsing research [2], [37], [38]. The process combines three complementary techniques to maximize recall and ensure practical applicability in large-scale CTI analysis. First, a semantic-based chunking method is applied to divide long CTI reports into coherent and manageable segments, similar to [2], which demonstrated that segmentation preserves contextual integrity and prevents extraction drift. Second, multiple LLMs—including GPT-4o, DeepSeek-V3, and LLaMA3—independently analyze each segment. Following the ensemble-style extraction principle widely adopted in recent LLM-based CTI studies [37], all IOC candidates identified by any model are retained to mitigate individual model bias and maximize recall. Finally, we perform multiple high-temperature inference passes to encourage output diversity, allowing the models to surface less explicit but contextually relevant IOCs, consistent with sampling strategies proposed in [38]. Together, these techniques form a practical and reproducible pipeline that provides sufficient IOC coverage for downstream regex generation and evaluation. We analyzed over 230,000 sentences and extracted more than 63,000 IOC candidates as inputs for subsequent regex
generation experiments. As shown in Table 1, 2,302 were registry keys, 12,195 were file paths, and 10,286 were command line arguments—representing the three regexapplicable IOC categories emphasized in Section 2.1.1. The remaining 38,820 entries fell into the “Other” category, which includes lower-value indicators such as IP addresses, domain names, hash values, and miscellaneous textual artifacts that are less suitable for regex-based operationalization. 5.1.2. Ground Truth Strings Dataset for Regex Validation. While CTI reports provide an excellent input data source for IOC extraction, validating our system’s processing results requires ground-truth information that is independent of these reports. However, currently available public datasets cannot provide the necessary ground-truth information for comprehensive validation. For example, [39] encrypts its content due to privacy concerns, and [40] does not contain sufficiently rich attack data. To validate whether the regexes generated by our system can be effectively applied in real-world environments, we utilized data from the MITRE ATT&CK Evaluation [12]. The MITRE ATT&CK Evaluation is an independent and rigorous assessment designed to test whether cybersecurity vendors’ products can detect and report various types of attacks. A total of 19 well-known cybersecurity vendors participated, including Bitdefender, Palo Alto Networks, Trend Micro, among others. In this evaluation, 10 well-known attack scenarios (such as APT29, LockBit, etc.) are replicated. In each scenario, there are multiple tactics ranging from Initial Access to Exfiltration, and each tactic may be executed using various techniques and multiple procedures. These vendors provide SIEM screenshots as evidence that they have successfully detected the execution of relevant TTPs. These screenshots contain numerous real-world file paths, registry keys, and command line arguments. We manually collected all the vendor screenshots and extracted the strings that related to file paths, registry keys, and command line arguments from each one. In total, we collected over 2400 strings across ten attack scenarios as ground-truth, with each attack scenario being a separate dataset. Table 2 shows the number of ground-truth strings in each dataset, as well as how many of those strings are file paths, registry keys, and command line arguments.
5.2. RQ1: Can IOCRegex-gen generate regular expressions in a scalable way We examine to what extent the regexes generated from the IOCs extracted from 3,156 CTI reports can cover the ground-truth strings in the MITRE ATT&CK Evaluation datasets, and each extracted IOC is guaranteed to have a corresponding generated regex. Since this study investigates whether our system can generate regexes that accurately retrieve relevant strings from large volumes of log data, we focus specifically on evaluating the accuracy of the regexes themselves. In Table 3, the “Number of matched strings” refers to the number of ground-truth strings in each dataset that can be matched by the generated regexes. The “Hit Rate” represents the proportion of these matchable ground-truth strings
Dataset
File path
Registry key
CLI args
Total
Enterprise2024 LockBit Enterprise2024 CL0P Turla Carbon Turla Snake Wizard Spider Wizard Sandworm APT29 scenario1 APT29 scenario2 FIN7 Carbanak
150 59 200 235 162 102 150 50 204 187
15 5 12 1 6 3 20 19 5 16
42 31 94 131 91 54 91 155 72 53
207 95 306 367 259 159 261 224 281 256
TABLE 2: Ground truth strings count by threat group and scenario relative to the total. The results demonstrate that regexes generated by our system achieve a 100% hit rate on groundtruth strings in the Enterprise2024 LockBit, Turla Snake, and APT29 Scenario2 datasets. In the remaining datasets, although not all ground-truth strings can be fully matched, the lowest hit rate remains 97.7%. Notably, none of the ground-truth strings were derived from any of the 3,156 CTI reports; instead, they originate from an independent source—the MITRE ATT&CK cybersecurity product evaluation results. The “Number of Unmatched Strings” represents the total number of ground-truth strings in the dataset that were not matched, and also specifies how many of those unmatched strings belong to file paths (Paths), registry keys (RegKeys), or command line arguments (CmdLines). And we can observe that most of the missed ground-truth strings fall within the “command line argument” category. Our analysis identifies two key characteristics shared by these unmatched strings. First, these cases commonly involve attacker-created executables—that is, custom executables dropped by adversaries during intrusion campaigns, rather than built-in Windows utilities. For instance, the ground-truth string C:\users\kmitnick.hospitality\appdata\loc al\adb156.exe //b //e:jscript sql-rat.js in APT29 Scenario1 represents an invocation of such a malicious payload. Because these executables are neither part of the standard system toolset nor previously seen in our graph database, the agent lacks sufficient contextual knowledge to model their associated command-line parameters—thus failing to generate matching regexes for this specific IOC type. Nevertheless, these attacker-created executables usually exhibit distinctive and traceable path patterns, such as anomalous directory locations (e.g., user profile or temporary directories) or randomized filenames. Even though the agent cannot always reconstruct the full command-line expression, it can still produce accurate regexes that capture the file path portion (e.g., C:\users\kmitnick.hospitality\appdata\loc al\adb156.exe), which remains useful for downstream detection.
Second, we discovered that many cmdlets actually used in real-world attacks are not documented in CTI reports. A cmdlet is a type of PowerShell command specifically designed to perform a particular task, such as accessing the file system, managing the registry, or querying the list of services. Attackers often leverage cmdlets to carry out their malicious objectives. For example, in the Wizard Spider case, the attacker uses the select-string cmdlet to search for text in strings and files, yet most CTI reports fail to document this usage. Consequently, our system cannot leverage these IOCs as intermediate results to generate the corresponding regexes. However, since we maintain a comprehensive inventory of Windows commands and PowerShell commands (including cmdlets), if a CTI report does include IOCs associated with any of those commands, our system can still use those IOCs to generate the corresponding regexes.
5.3. RQ2: False Positives of the Generated Regular Expressions Some generated regexes may incorrectly match groundtruth strings they should not target. Each such incorrect match constitutes a false positive. We determine false positives by comparing the capture groups in each generated regex against those in its matched groundtruth string. Specifically, a match is considered a false positive if the regex successfully matches a groundtruth string whose capture groups differ from those of the IOC used to generate the regex. For example, the IOC C:\Windows\System32\certutil.exe has capture groups Windows and System32. The corresponding regex correctly matches a ground truth string such as c:\windows\system32\pscp.exe, which shares identical capture groups with the original IOC. Conversely, if it matches a ground truth string such as c:\users\pam\desktop\rcs.3aka3.doc, whose capture groups differ, the match is considered a false positive. To systematically evaluate false positives, we first need to determine the capture group for each ground-truth string. Following the definitions of “capture group” and “non-capture group” mentioned in Section 3.1, we annotate the capture group for every ground-truth string. For each generated regex, we check which ground-truth strings it matches and count those whose capture groups differ from the regex as false positives. Then calculate the proportion of these IOCs as the false positive rate, and finally compute, for each dataset, the average false positive rate of regexes for IOCs that match ground truth strings. We formalize this approach formally. Let D represent the set of all ground-truth strings in the current dataset, and G(s) denote the capture group for any string, s ∈ D. Let IOCk denote the source IOC from which k-th regular expression Rk is generated. We then define Gk = G(IOCk ) as the capture groups in that IOC. Define the set of ground-truth strings that Rk (s) can match as Mk = s ∈ D : Rk (s) matches , where Nk = |Mk | represents the total number of match strings. Using FPk = { s ∈ Mk : G(s) ̸= Gk } , we can calculate how many ground-truth strings that Rk should not
Total # of Unmatched # of Matched ground-truth (Total/Paths/RegKeys/ Hit Rate (%) Avg. FP Rate (%) Strings strings CmdLines)
Dataset Enterprise2024 LockBit Enterprise2024 CL0P Turla Carbon Turla Snake Wizard Spider Wizard Sandworm APT29 scenario1 APT29 scenario2 FIN7 Carbanak
207 95 306 367 259 159 261 224 281 256
207 94 304 367 254 157 255 224 278 254
0/0/0/0 1/0/0/1 2/0/0/2 0/0/0/0 5/1/0/4 2/1/0/1 6/0/0/6 0/0/0/0 3/1/1/1 2/1/0/1
100.0 98.9 99.3 100.0 98.1 98.7 97.7 100.0 99.0 99.3
0.0 0.0 1.5 1.2 0.5 1.3 0.9 0.9 1.4 0.3
TABLE 3: Experimental results of whether regex generation was successful (RQ1) and the false positive rate of the generated regex (RQ2). match but it actually does match (false positives). The false positive rate for regex Rk is then: { s ∈ Mk : G(s) ̸= Gk } FPk = . (2) FPR Rk = Nk |Mk | To obtain the overall performance metric for each dataset, we compute the average false-positive rate across all regexes that successfully match at least one ground-truth string: 1 K FPR Rk (3) FPR = ∑ M k=1 Nk ≥1
where M represents the number of regexes that match ground-truth strings in the dataset. Our evaluation demonstrates that all false-positive rates remain 1.5% or lower, as shown in Table 3. Analysis of the regexes with relatively higher false-positive rates reveals a consistent pattern: such cases occur when attackers deliberately place malicious executables in non-native Windows directories. For instance, paths like c:\dumps\microsoft.activedirectory.webser vices.exe.4652.dmp.sk12uyqzk do not conform to typical Windows directory structures, causing regexes targeting standard system folders to miss them unless the matching patterns are broadened. However, these false positives are readily identifiable and resolvable in real SOC workflows. The matched paths usually deviate from legitimate Windows directory conventions—appearing in user-created folders, temporary dump locations, or application-specific cache directories—making they could be immediately distinguishable by a set of heuristic rules. During log triage, analysts can use these rules to filter such entries in seconds by verifying directory provenance or applying simple whitelists of common benign paths.
5.4. RQ3: Over-Generalization Assessment In RQ3, we investigate whether the generated regexes exhibit over-generalization. We demonstrate that the regexes produced by our system meaningfully incorporate capturegroup information rather than relying on overly simplistic patterns such as (?i).*cmd.* or (?i).*HKCU.*.