Merging Cyber Threat Intelligence Through Retrieval-Augmented Generation and Small Language Models for Rich Threat Representation
arXiv:2609.07280v1 [cs.CR] 7 Sep 2026
Nicola Deidda1,2[0009−0005−7309−5703] , Leonardo Regano1[0000−0002−9259−5157] , Alessandro Sanna1[0000−0002−0610−7736] , Davide Maiorca1[0000−0003−2640−4663] , and Giorgio Giacinto1,3[0000−0002−5759−3017] University of Cagliari, Cagliari, Italy [email protected] IMT School for Advanced Studies Lucca, Lucca, Italy [email protected] 3 Consorzio Interuniversitario Nazionale per l’Informatica, Rome, Italy 1
2
Abstract. Modern cybersecurity operations rely on Cyber Threat Intelligence (CTI) collected from heterogeneous sources, including semistructured threat representations, Indicators of Compromise (IoCs), and narrative technical reports. However, these artifacts are often insufficient in isolation to reconstruct how an attack unfolds, under which conditions each step is feasible, and which traces it leaves behind. In practice, analysts must manually correlate partial evidence scattered across multiple and only partially structured sources, delaying the design of effective prevention, detection, and response actions. To address this gap, we propose an automated pipeline that derives an actionable representation of a cyberattack from heterogeneous CTI sources. The pipeline combines a Retrieval-Augmented Generation (RAG) architecture, used to retrieve step-relevant evidence from dispersed documents, with a locally deployable Small Language Model (SLM), used to consolidate such evidence and infer missing operational details. Starting from a semi-structured threat representation and auxiliary CTI documents, the pipeline produces an enriched Attack Graph that captures a coarse, tactic-aligned progression of the attack and annotates each step with an enriched description, explicit pre-conditions, and explicit post-conditions. This representation supports prevention by exposing execution requirements, detection by highlighting observable traces, and response by clarifying the temporal progression of the attack. Then, due to the lack of validated datasets with ground-truth information on the temporal evolution of real-world attacks, we test the complete pipeline on 10 real-world case studies spanning multiple threat types, including backdoors and staged downloaders delivered via phishing. A manual assessment across 10 real-world case studies provides initial evidence that the generated Attack Graphs are consistent with expected attack progressions, indicating that the proposed approach can support analysts by consolidating dispersed CTI evidence into a structured and actionable view of attacks. Keywords: Cyber Threat Intelligence · Small Language Models · Attack Graphs · Attack Flow · Preconditions · Postconditions
2
Deidda et al.
1
Introduction
Modern defense mechanisms rely on large-scale data to model, analyze, and counter cyber threats [23]. In this context, integrating Cyber Threat Intelligence (CTI) processes into organizational operations improves cybersecurity posture [25]: these processes consume CTI feeds, enrich and analyze the data, and disseminate derived artifacts. However, collected CTI (e.g., technical reports, structured files, blog articles) still requires substantial manual processing to become operationally actionable [12][30]. In practice, no single CTI artifact provides a complete operational view of an attack. Indicators of Compromise (IoCs) expose observable signs of compromise, while standards such as STIX [1] define entities, attributes, and relations, thus supporting intelligence sharing. However, these artifacts are not sufficient to reconstruct how an attack unfolds over time, under which conditions each step is feasible, and which traces or system changes it produces. As a consequence, analysts often reconstruct the execution flow manually from heterogeneous sources, then derive each step’s pre-conditions and post-conditions (e.g., required privileges and observable artifacts) [11], a costly and error-prone task that delays the design of effective prevention, detection, and response actions. In this work, we do not infer a fully ordered execution trace; rather, we derive a tactic-aligned representation of coarse attack progression and enrich its steps with grounded operational details. Such information is typically represented through Attack Graphs, a widely used method for helping security practitioners understand the temporal evolution of cyberattacks [15]. Several works model cyber threats from unstructured CTI using Large Language Model (LLM)-based pipelines and knowledge graphs [8, 33, 17, 7, 38]. However, these representations are typically entity-centric and do not explicitly emphasize the execution flow, nor do they provide step-level pre-conditions and post-conditions that support feasibility assessment and investigation. Our focus is instead on step-level operational semantics: starting from heterogeneous CTI, we aim to reconstruct a coherent attack progression and enrich each step with the information required to make that representation directly useful to analysts. Motivated by this gap, we study the following research questions: RQ1 Is it possible to leverage Natural Language Processing (NLP) and Artificial Intelligence (AI)-based techniques to extract, from heterogeneous technical sources, coherent Attack Graphs that describe the temporal evolution of an attack? RQ2 Is it possible to leverage NLP and AI-based techniques to enrich each attack step of an Attack Graph with coherent descriptions, pre-conditions, and post-conditions, while supporting the continuous integration of data? RQ3 Can these technologies generalize this inference process across diverse realworld cyber threats belonging to different threat families, involving different architectures, and Tactics, Techniques, and Procedures (TTPs)? To address these questions, we propose a pipeline that, starting from a heterogeneous set of documents describing a cyber threat, derives a tactic-aligned
Merging CTI Through RAG and SLM for Rich Threat Representation
3
attack representation and enriches each step with grounded descriptions, preconditions, and post-conditions: 1. We derive a coarse attack progression by leveraging a Small Language Model (SLM) [2] over documents stored and retrieved via a Retrieval-Augmented Generation (RAG) architecture. The result is modeled as an Attack Graph [20] extended with nodes defined as Milestones and Attack Steps, where Milestones are tactic-aligned execution checkpoints and Attack Steps are atomic attacker actions. 2. We enrich the graph by grounding Attack Steps in evidence from heterogeneous documents and by inferring, for each Attack Step, the actions preconditions and resulting post-conditions. Pre-conditions support preventionoriented prioritization by filtering infeasible steps, while post-conditions provide concrete indicators for detection and post-incident investigation. 3. We empirically evaluate the pipeline on 10 real-world case studies spanning multiple threat families to provide feasibility evidence across diverse scenarios. To the best of our knowledge, no public datasets map cyberthreats to attack flows with step-level pre-conditions and post-conditions. More broadly, the scarcity of validated ground-truth data on cyberattacks remains a current issue in CTI research. Krašovec et al. [13] highlighted the volatility of ground truths in CTI, which depends on expert analysis and mapping of technical reports. Furthermore, intelligence should be manually validated; this is a time-consuming, error-prone process. Given the current absence of large amounts of validated data, we validate the pipeline components individually. Then, we empirically verify the complete execution across 10 case studies. The remainder of the paper is organized as follows: Section 2 introduces the core concepts underlying the pipeline; Section 3 details the proposed methodology; Section 4 presents experimental results; Section 5 discusses findings and limitations; Section 6 compares related literature; and Section 7 concludes with future research directions.
2
Background
Our pipeline processes heterogeneous CTI sources, which are predominantly textual (e.g., technical reports and blog articles). To ground generation in this evidence, we rely on retrieval-centric methods, particularly RAG, whose effectiveness depends on document chunking and retrieval quality. 2.1
Retrieval-Augmented Generation
RAG [6, 31] combines information retrieval and text generation by conditioning a language model on external context retrieved from a document store. A typical RAG system comprises (i) a retriever that selects relevant text chunks from a knowledge base (e.g., a vector database) and (ii) a generator (an LLM or SLM) that synthesizes the retrieved context into an output. In CTI, RAG is
4
Deidda et al.
useful because evidence about an attack step is often dispersed across semistructured threat representations, technical reports, and auxiliary artifacts. By retrieving and consolidating these fragments, RAG grounds the generation of step-level descriptions, pre-conditions, and post-conditions in source evidence. Its effectiveness, however, remains sensitive to chunking choices, index maintenance, and retrieval noise. Chunking [14] splits documents into smaller units to support efficient retrieval of semantically coherent evidence. In RAG pipelines, the chunking strategy directly affects recall and precision: small chunks may separate related content, whereas large chunks may mix relevant and irrelevant information. Common approaches include fixed-size chunking, which is simple but may fragment meaning, and semantic chunking, which aims to preserve coherent boundaries. Chunk overlap can further mitigate boundary effects. 2.2
Hybrid Search
Hybrid search [3] combines lexical retrieval [4, 24], which ranks documents by matching the query terms that explicitly appear in the text, with embeddingbased semantic search [9], which represents both document content and the query terms in a dense vector space and ranks documents by semantic similarity between their content and the query terms. This combination is well-suited to CTI, where some elements are best matched exactly (i.e., through lexical retrieval), while others are described with varying terminology across sources (i.e., through embedding-based semantic search). Lexical methods provide high precision for technical artifacts such as ATT&CK identifiers, malware names, file paths, domains, registry keys, or service names, whereas semantic retrieval improves robustness to paraphrases and vocabulary mismatch. In practice, both scores can be computed independently and combined (e.g., via weighted scoring) to rank candidate chunks for generation.
3
Methodology
We design a local-first pipeline that transforms a semi-structured threat representation (e.g., STIX) and heterogeneous CTI documents into an enriched Attack Graph capturing attack evolution. The pipeline reduces analyst effort by grounding enrichments in retrieved evidence and using an on-device SLM to consolidate information across sources. Using an on-device SLM addresses organizational constraints on sharing sensitive data with external platforms and targets deployments with limited computational resources. 3.1
Pipeline Overview
Figure 1 summarizes the proposed pipeline. Inputs and output. Let D be a set of heterogeneous CTI documents and T a semi-structured threat representation. The pipeline outputs an enriched attack
Merging CTI Through RAG and SLM for Rich Threat Representation
5
HETEROGENEOUS DOCUMENTS RAG Architecture chunking VECTOR STORE
SLM 5
context & prompt
1
2
BM25
embedding
CHUNKS
ENSEMBLE RETRIEVER
response
PRE CONDITIONS
6
3
fetch relevant chunk
POST CONDITIONS
chunks
4
ATTACK STEPS DESCRIPTIONS
REFINEMENT
OUTPUT
Attack Flow Reconstruction THREAT REPRESENTATION 0
preliminary modelization USER
Fig. 1: High-level overview of the proposed pipeline. Documents chunks are ingested inside the RAG architecture. The SLM uses the retrieved chunks to infer the pre-conditions, post-conditions, and the enriched description for each Attack Step.
graph G = (V, E), where V = Vms ∪ Vas : each milestone v ∈ Vms represents a MITRE ATT&CK tactic, and each attack-step node v ∈ Vas represents a technique instance derived from T . The resulting structure captures a coarse, tactic-aligned progression of the attack rather than a fully ordered execution trace. Each v ∈ Vas is annotated with an enriched description, a set of pre-conditions, and a set of post-conditions inferred from evidence retrieved from D. Given T and D, (1 ) we normalize documents to text and split them into semantically coherent chunks, (2 ) index them for hybrid retrieval, and, for each attack step derived from T , (3,4 ) retrieve the most relevant chunks, (5 ) assemble a bounded context, and (6 ) query the SLM to produce an enriched step description and associated pre-conditions/post-conditions. The resulting annotations are attached to G and can be optionally refined in a post-processing stage.
3.2
User-provided Threat Representation and Heterogeneous Documents
This section summarizes assumptions on the user-provided threat representation and the heterogeneous documents ingested by the pipeline. Our implementation uses STIX, which is widely used in organizations, but alternative formats (e.g., MISP) may be supported with limited engineering effort. We assume: (i ) the STIX file is valid and standard-compliant; (ii ) auxiliary documents are already collected and sanitized to remove closed-source or sensitive content [36]; and (iii ) sources are expert-produced and trustworthy, i.e., we
6
Deidda et al.
exclude knowledge-base poisoning [28]. Although poisoning is out of scope, prior work proposes ways to assess CTI provider quality [21, 35, 26]. From the STIX file we extract (i ) the Malware object: its name is used to generalize action titles and descriptions; if multiple instances exist, the user selects the target one (e.g., in multi-stage campaigns); (ii ) each Attack Pattern object: we collect name, description, and kill chain phase [10]. MISP provides analogous entities (notably attack-pattern and malware)4 . We use the kill chain phase metadata to obtain a coarse sequencing reference for attack evolution. The heterogeneous documents are normalized to text (PDF/HTML) and chunked using the semantic chunking strategy described in Section 2.1. We explicitly parse semi-structured JSON, XML, YAML, and CSV; all other formats are processed as text using the same chunking procedure. Each chunk stores its textual content and source-file metadata, enabling traceability to the originating document. 3.3
RAG Architecture
The enrichment stage relies on retrieving supporting evidence from D, using stateof-the-art NLP and context retrieval techniques. We implement a hybrid retriever that combines lexical ranking (Best Matching 25 (BM25)) with embeddingbased retrieval over a Facebook AI Similarity Search (FAISS) vector database [5]. Lexical retrieval. We configure a BM25 retriever to return the top-kbm25 chunks for a query. Vector retrieval. We use FAISS to embed each chunk, and index the resulting embeddings, for approximate nearest-neighbor search. In our implementation, FAISS uses an L2 index; given a query embedding, we retrieve the top-kvdb nearest chunks by Euclidean distance. Embeddings are computed using the llama3.1 embedding model. Ensemble scoring. Given a query q and candidate chunk c, we compute s(c) = wbm25 · BM25(q, c) + wvdb · sim(eq , ec ), where eq , ec are embeddings and (wbm25 , wvdb ) ∈ [0, 1] controls the ensemble. Query construction. For each Attack Step, we build a retrieval query by combining the malware name, the MITRE technique name, and the original description available in the threat representation. This query formulation allows the retriever to match both exact technical artifacts and semantically related descriptions across the heterogeneous document collection. The top-ranked chunks are then deduplicated and assembled into the bounded context provided to the SLM for step enrichment. 3.4
Enrichment Process and Attack Graph
The pipeline constructs an Attack Graph aligned with MITRE ATT&CK. We group Attack Pattern instances by ATT&CK tactic: each tactic corresponds to a 4
https://www.misp-project.org/objects.html
Merging CTI Through RAG and SLM for Rich Threat Representation Attack MITRE Step Technique
a2
a1
T1543.003
a3
a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12
T1036.005 T1497.003 T1497.001 T1497.002 T1027 T1140 T1564.003 T1070.010 T1574.001 T1055.012 T1218.011
a4
a13
T1049
a14 a15 a16
T1071.001 T1008 T1571
7
a5 a6 Start
m1
a1
m2
a7
a14 m3
a13
a8
m4
a15
End
a16
a9 a10 a11 a12
Fig. 2: Attack Graph representing the coarse, tactic-aligned progression of the cyber attack described in the first case study. The table on the left lists the MITRE Technique IDs for each Attack Step. Legend:
Attack Step
Milestone
milestone (Milestone), and each associated technique to an attack step (Attack Step). The resulting graph organizes attack steps under milestones, where each step represents an atomic attacker action against the victim asset. Graph semantics. Milestones are ordered according to the kill chain phase metadata associated with Attack Pattern objects, which provides a coarse sequencing reference. Within a milestone, attack steps are treated as an unordered set unless the input representation explicitly supports finer-grained sequencing. Accordingly, edges in G capture tactic-level progression and step membership rather than prerequisite or causal relations between techniques. Step enrichment. For each Attack Step, we retrieve evidence chunks from the RAG architecture and build a bounded context. The SLM is prompted to output: (i ) a concise enriched description grounded in the retrieved context; (ii ) a list of pre-conditions describing feasibility requirements (e.g., privileges, environment constraints, dependencies); and (iii ) a list of post-conditions describing direct observable consequences (e.g., artifacts, configuration changes, network connections). To limit generic statements, pre-conditions/post-conditions are constrained to short, atomic items, and the model is instructed to rely only on the provided context. Running example. To describe how the pipeline works, we introduce a case study focusing on Goofy Guineapig, a Windows backdoor that persists via a Windows service, communicates with a remote C2 server via multiple channels, and employs defense-evasion techniques. We first retrieved the STIX file from
8
Deidda et al.
Table 1: pre-conditions and post-conditions for the first Attack Step pre-conditions
post-conditions
1. The malware has successfully infected a target system running Windows. 2. The malware can access system files and registry entries related to services. 3. A Windows operating system is present.
1. Network traffic logs showing communication with attackercontrolled server. 2. Potential for further malicious activity on the compromised system. 3. System instability and performance degradation. 4. New user account created or existing account privileges elevated. 5. Data exfiltration to attackercontrolled server. 6. Traces of data transfer to external storage devices. 7. Modified system registry entries. 8. Compromised system with persistent backdoor access. 9. Altered system configuration files. 10. Hidden files and folders containing malware components. 11. Event log entries indicating suspicious activity, such as service creation or process execution.
the National Cyber Security Centre (NCSC) website5 and complemented it with external sources. The document pool consisted of (a) a 28-page PDF detailing the malware behavior, (b) a CSV containing additional Indicators of Compromise (IoCs), and (c) five YAML files offering contextual information on specific malicious actions. The chunking process produced 172 chunks. Figure 2 shows the visualization of the Attack Graph generated. We labeled each Attack Step with the corresponding TTP to highlight that our pipeline produces a logic representation of the attack flow. These represent the mandatory actions required to move from one pivot point to the next. For instance, in the image, to move from the first to the second pivot point, the Attack Steps from a1 to a12 must be executed successfully. One of the key points of our proposal is the capability to infer step-level preconditions and post-conditions grounded in the retrieved evidence. Considering the first Attack Step of the first Milestone, Table 1 reports a compact example. For clarity, this is the only Attack Step related to that Milestone, and its 5
https://www.ncsc.gov.uk/static-assets/documents/ malware-analysis-reports/goofy-guineapig/NCSC-MAR-Goofy-Guineapig-stix. json
Merging CTI Through RAG and SLM for Rich Threat Representation
9
Table 2: Comparison between the original and enriched description Original Description
Enhanced Description
Goofy Guineapig maintains persistence as a Windows service.
A Windows service is established by the malware designated as "Goofy Guineapig" to ensure persistence within the system. The service, named "GoogleUpdate," is configured with a display name that likely mirrors the service name. The creation and configuration of this service are facilitated through the execution of shellcode originating from a trojanized GoogleUpdate installer. This shellcode leverages the rundll32 command to specify the service parameters, including its name, display name, startup type, and the path to the malware’s executable file as the service binary. The elevated privileges inherent in Windows services grant malware access to sensitive system resources and data, while their background operation can render them less conspicuous to security software and users.
associated Technique is “Create or modify system process: Windows service” (T1543.003). The inferred pre-conditions capture feasibility constraints, such as OS dependence and required access to service-related resources, while the post-conditions describe direct artifacts and effects that may support validation and post-incident investigation. Their quality depends strongly on the specificity of the retrieved evidence: concrete technical details tend to yield more precise consequences, whereas sparse or high-level sources may lead to generic statements. Beyond pre-conditions/post-conditions, the Attack Graph also enriches each Attack Step with a consolidated description. Table 2 shows that the enriched description expands the original STIX text by integrating details from the heterogeneous document pool, thereby improving completeness.
4
Experimental Results
In this section, we validate the pipeline described in Section 3. All the experiments, excluding the evaluation of our approach on a set of case studies reported at the end of this Section, have been performed on a system with 250 GB of RAM and an NVIDIA RTX 5000 Ada Generation 32GB. The machine runs Ubuntu 22.04.5 LTS and Ollama 0.11.11. To design a technology-agnostic solution, we developed a modular pipeline. The SLM used runs locally on Ollama6 , a lightweight framework. The choice was driven by ease of use and the updated model pool. Other possible alternatives 6
https://ollama.com/
10
Deidda et al.
0.9
BM25 − Precision BM25 − Recall BM25 − F1
0.8 0.7
0.9 0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0.0
135 10 15 20
30
40
50
kbm25
100
(a) BM25 retriever performance when kbm25 ∈ [1, 3, 5, 10, 15, 20, 30, 50, 100]. Precision, Recall, F1.
VDB − Precision VDB − Recall VDB − F1
0.8
0.0
135 10 15 20
30
40
50
kvdb
100
(b) Vector DB retriever performance when kvdb ∈ [1, 3, 5, 10, 15, 20, 30, 50, 100]. Precision, Recall, F1.
Fig. 3: Retriever performance as the top-k hyperparameter varies.
have been explored, such as the models available at Hugging Face7 , which provides a similar solution. 4.1
Preprocessing Steps
To choose a chunking strategy for our RAG architecture, we followed the evaluation by Smith and Troynikov [27] and adopted a ClusterSemanticChunker configured with a maximum chunk size of 200 tokens and no overlap. This choice preserves semantically related content while keeping retrieved context compact. Embeddings for chunking and similarity computations are produced with all-MiniLM-L6-v2 8 , a lightweight sentence-transformer that outputs 384dimensional dense vectors. 4.2
RAG Hyperparameters
The RAG architecture comprises three components (Section 3.3), each with dedicated hyperparameters. The BM25 retriever uses kbm25 ∈ N, the number of documents returned by the BM25 ranking. The vector-database retriever uses kvdb ∈ N, the number of documents retrieved via embedding similarity search. The ensemble retriever is parameterized by (wbm25 , wvdb ), where wbm25 , wvdb ∈ R+ and wbm25 + wvdb = 1, which control the contribution of each retriever to the final ranking. We tuned these hyperparameters on SQuAD [22] as a generic retrieval sanity check, using question–context pairs with ground-truth answers. Because SQuAD is not CTI-specific, the resulting values should be interpreted as practical default 7 8
https://huggingface.co/ https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
Merging CTI Through RAG and SLM for Rich Threat Representation
Precision
Recall
11
F1
0.75 0.50 0.25 0.00
(0.2,0.8) (0.4,0.6) (0.5,0.5) (0.6,0.4) (0.8,0.2)
(wbm25 , wvdb )
Fig. 4: Evaluation of the weights (wbm25 , wvdb ) for the ensemble retriever settings rather than task-optimal hyperparameters for CTI. After removing duplicate contexts, we retained 442 documents, each containing a single question. Our evaluation pipeline (i) chunks the documents (968 chunks), (ii) indexes them in the RAG components, and (iii) queries the retrievers with the corresponding questions. We first tested BM25 and the vector retriever independently and measured effectiveness using mean Context Precision and mean Context Recall over the full question set. Based on the results in Figure 3a and Figure 3b, we selected kbm25 = 50 and kvdb = 30 as stable defaults. With these fixed, we evaluated ensemble weighting by combining both retrievers with different (wbm25 , wvdb ) values (Figure 4) and ultimately adopted equal weights, i.e., (0.5, 0.5). 4.3
External Documents and STIX Files
To gather the external documents and the STIX files, we relied on the UK NCSC9 repository. This source provides an analysis of different malware. Hence, we used these analyses as case studies to validate our pipeline. For each case study, we built a document pool by searching on blogs and CTI platforms. We relied on the NCSC files because we assume the correctness of their content. Although the trustworthiness and quality assessment of CTI feeds has been widely discussed in the literature [18] [29], these are outside the scope of this proposal; thus, our assumption is primarily based on the entity’s public reputation. In addition, these files provide a consistent number10 of case studies for evaluation, and the documents include summaries that can be used as references. Among the reports provided by the source, we selected 10 for further enrichment, feeding for each attack the set of documents describing it to the pipeline. Thus, we obtained an Attack Graph for each case study, which we manually verified for coherence with respect to the original documents describing the attack. In particular, given the attack described in the documents, we manually verified the correctness of each Attack Step and edge in the Attack Graph (RQ1). Furthermore, for each Attack Step in each Attack Graph, we verified the correctness 9 10
https://www.ncsc.gov.uk/ https://www.ncsc.gov.uk/section/keep-up-to-date/ malware-analysis-reports
12
Deidda et al.
of each pre-condition and post-condition (RQ2) and that they are grounded in the heterogeneous input documents. These case studies are used to evaluate the pipeline since they cover different cyber threat families, thus allowing us to evaluate the pipeline’s generalization capabilities (RQ3). Table 3 summarizes these cases. We presented the first as a running example in Section 3.4. We provide the results on the other use cases and the pipeline source code in a public repository11 . This has been done to ensure reproducibility of the results and to further test our proposal. To assess the feasibility of adopting our approach in resource-constrained organizations (e.g., Small and Medium Enterprises), we performed the case study experiments on a commercial laptop featuring 32 GB of RAM and an NVIDIA GeForce RTX 4060 8GB. We run Ollama 0.11.11 on Windows 11 Enterprise.
Table 3: Summary of the proposed case studies with a brief description defining the cyber threat Case
Description
1
Persistent Windows backdoor with HTTPS C2 communications. MacOS malware using a custom data encoding algorithm over HTTPS. Telegram Bot API–based backdoor with file download and execution capability. Cisco IOS malware collecting device information and enables backdoor access. A Windows remote access tool. Malware targeting Fortinet devices. Native ELF shared object providing backdoor access on Sophos XG firewalls. Malware running on Windows within the Outlook process to exfiltrate sensitive data. Persistent loader and backdoor with XOR-encoding for the C2 channel. A staged downloader targeting Windows, delivered via spear-phishing.
2 3 4 5 6 7 8
9 10
# of Documents
# Attack Step
7
16
7
9
9
7
9
11
5 12 13
12 12 12
11
8
3
9
6
14
During these evaluations, we executed the application using the default parameters described in the previous sections, noting that all values can be adjusted through the provided configuration file. The SLM employed was gemma2:9b, used 11
https://anonymous.4open.science/r/ActionableRichThreatRepresentationLLM
Merging CTI Through RAG and SLM for Rich Threat Representation
13
for inferring pre-conditions and post-conditions and for assembling the enriched descriptions. This model was chosen for its lightweight nature, which reduces computational overhead and execution time. Similarity computations relied on the all-MiniLM-L6-v2 embedding model, selected for its efficiency and broad library support, as discussed in Section 4.1.
5
Discussion
Figure 2 illustrates the enriched Attack Graph produced by our pipeline and highlights the tactic-aligned structure of the resulting representation. Unlike classic attack graphs centered on infrastructure entities (e.g., hosts and services) and reachability relations, our formulation models attack evolution: nodes represent either attack steps (Attack Steps) or pivot points (Milestones), and edges capture coarse progression between campaign stages. The resulting graph describes a coarse, tactic-aligned progression of the attack and was manually verified for node and edge correctness (RQ1). Each Attack Step is enriched with execution requirements (pre-conditions) and expected effects (post-conditions), making the representation more actionable for analysis and investigation. In particular, pre-conditions highlight feasibility constraints and dependencies, whereas postconditions expose observable consequences useful for detection, forensic validation, and incident response. The pipeline generates the full graph, including Milestones, Attack Steps, and their enrichments, using the same prompting strategy across all case studies (RQ2). To quantify enriched-description quality, we compute BERTScore [37] against references assembled from external documents. We use BERTScore only as a proxy for the similarity of enriched textual descriptions to reference descriptions; it does not directly measure graph correctness, grounding, or analyst utility. Because SLM outputs are non-deterministic, we repeat enrichment ten times and report the resulting F1 distributions (Fig. 5). The results indicate stable similarity to the references across runs for most case studies, supporting generalization across diverse threats (RQ3).
Evaluation F1 Scores
0.855 0.850
F1 Score
0.845 0.840 0.835 0.830 0.825 0.820
1
2
3
4
5
6
Case #
7
8
9
10
Fig. 5: Box plots of BERTScore F1 values for enriched descriptions across the 10 case studies.
Deidda et al. Evaluation Execution Times
Execution Time (s) per Attack Step
14
325
2750
Evaluation Execution Times per Attack Step
Execution Time (s)
300
2500
275
2250
250
2000
225
1750
200
1500
175
1250 1
2
3
4
5
6
Case #
7
8
9
10
150
1
2
3
4
5
6
Case #
7
8
9
10
(a) Box plots of the execution times for each (b) Box plots of the execution times per case study. Attack Step for each case study.
Fig. 6: Execution time distributions across the 10 case studies.
Figure 6 reports end-to-end execution times across the 10 case studies and per-Attack Step costs. Runtime scales with the number of Attack Steps and the size of the retrieved context; Case 8 is an outlier, consistent with increased generation effort during enrichment. Finally, we empirically verified the Attack Graph structure by reconstructing the attack progression from the STIX file and the document pool and comparing it with the generated milestone ordering and step assignments. In the Goofy Guineapig example, the milestones correspond to consecutive campaign stages, which improves readability by grouping techniques into higher-level checkpoints. Limitations and failure modes. The pipeline inherits typical RAG failure modes. Retrieval misses may omit key evidence and produce incomplete enrichments; retrieval noise may introduce irrelevant context and lead to generic pre-conditions or post-conditions; conflicting sources may yield inconsistent descriptions; and sparse evidence may lead to plausible but weakly supported outputs. In operational settings, each enrichment should therefore be associated with its provenance, such as retrieved chunk identifiers and source references, to support analyst auditing and conservative fallback behaviors (e.g., emitting UNKNOWN when evidence is insufficient). 5.1
Threats to Validity
Following Wohlin et al. ’s validity taxonomy [32], we discuss threats to construct, internal, external, and conclusion validity. Regarding construct validity, the main construct is the correctness and usefulness of the enriched Attack Graph. However, manual verification of graph structure and step enrichments may introduce subjectivity, since no benchmark with step-level ground truth is currently available. We assess enrichment quality using BERTScore against references manually assembled from external documents; although BERTScore captures semantic similarity, it does not fully measure actionability, completeness, faithfulness, or analyst utility, and it is sensitive to reference wording. For retriever tuning, we compute mean context precision
Merging CTI Through RAG and SLM for Rich Threat Representation
15
and recall on SQuAD-derived question–context pairs, providing a controlled benchmark of retrieval performance. For internal validity, document pools were assembled to support the analyzed threats, likely reducing ambiguity compared with open-world CTI settings containing more irrelevant or noisy material. Model non-determinism is another threat [19]: the SLM outputs vary across runs, which we partially mitigate by repeating enrichment when computing BERTScore. We also adopted a small local model to reflect organizations with limited computational resources, although larger local models may be feasible when infrastructure permits. Regarding external validity, the evaluation covers 10 case studies across multiple threat families, but it does not capture the full breadth and noisiness of operational CTI ecosystems, such as multilingual reports, fragmented social media, or organization-internal intelligence. For conclusion validity, our results remain largely descriptive, combining manual checks, BERTScore, and execution-time measurements over a limited number of case studies. In the absence of an established ground truth for enriched attack-flow graphs with pre- and post-conditions, consistency across cases should be interpreted as feasibility evidence rather than definitive proof of general performance.
6
Related Works
To the best of our knowledge, prior work has not combined a RAG architecture, a local SLM, and a graph-based representation with explicit pre-conditions and post-conditions to derive an actionable view of coarse attack progression from CTI reports. The problem of modelling a cyber threat with a graph representation has been addressed by several works in the literature. Hu et al. [8] used a fine-tuned LLM to perform topic classification and entity–relationship extraction from narrative CTI. The extracted knowledge is used to construct the cyber threat’s knowledge graph. This proposal significantly differs from our methodology. First, our pipeline does not require fine-tuning a model, thus reducing the computational resources and data required. Fine-tuning an LLM is a resource-intensive task [33]; indeed, our approach demonstrated that relatively small models can achieve crucial results with a proper data ingestion pipeline. Moreover, the knowledge graph proposed in [8] models the cyber threats as entities and relationships. Instead, our revised Attack Graph focuses on coarse attack progression and on operationally relevant information, such as pre-conditions and post-conditions. Similarly, Liu and Zhan [17] focus on constructing a knowledge graph in which cyber threats are represented as entities and relationships. In addition, they rely on a closed-source model to extract data from CTI reports. Besides the difference in graph structure and goal, our pipeline differs in the model used. We leveraged an open model, thus improving data protection, particularly concerning sensitive data and closed-source intelligence.
16
Deidda et al.
Xu et al. [34] proposed IntelEx, an LLM-based solution to process unstructured CTI reports to extract TTPs, supported an LLM-as-a-Judge strategy [7] to mitigate inaccuracies. This proposal leverages LLMs and RAG to extract information from CTI sources. However, our proposal focuses on providing an enriched representation of attack progression rather than only extracting TTPs. Furthermore, the proposal in [34] relies on commercial LLMs. The proposal by Zhang et al. [38] leverages LLMs to construct an Attack Graph based on the cyber threat’s TTPs and extracted events. This contribution does not propose the coarse attack progression or crucial information such as the pre-conditions of the Attack Steps. The event extracted by the framework by Zhang et al. details the action corresponding to each MITRE ATT&CK Technique and the involved resource. However, the requirements for executing these actions are not provided. Instead, our methodology explicitly focuses on providing analysts a set of pre-conditions to assess when evaluating the cyberattack feasibility. Moreover, understanding the execution flow helps determine the feasible and infeasible stages of the attack, thereby supporting prioritization and post-incident response.
7
Conclusion and Future Work
In this article, we proposed a pipeline to infer an enriched representation of coarse cyber attack progression from heterogeneous CTI sources. By transforming fragmented CTI into a structured, tactic-aligned representation enriched with step-level operational details, the pipeline aims to reduce the analyst effort required to move from threat reporting to prevention, detection, and response planning. To provide this representation, we extended the foundational concept of Attack Graph by defining two node types: the Milestones and the Attack Steps. The former represents a pivot point in the attack flow. The latter represents the actions performed by the attacker. Furthermore, we provided the Attack Graph with the attack pre-conditions and the post-conditions. Then, each Attack Step contains an enriched description. The pipeline is powered by a RAG architecture in which retrieved evidence provides the context for the SLM response. The SLM executes locally, so closedsource intelligence is not shared with third parties, enforcing data protection. As future work, we plan to improve the pipeline’s reliability and robustness, particularly with respect to inference faithfulness and evidence grounding. We will evaluate recent approaches to improve reliability [16] in our pipeline. We also plan to exploit pre-conditions to evaluate attack feasibility, and post-conditions to drive incident response operations, support detection engineering, and facilitate post-incident investigation. Disclosure of Interests. The authors have no competing interests to declare that are relevant to the content of this article
Merging CTI Through RAG and SLM for Rich Threat Representation
17
References 1. Barnum, S.: Standardizing cyber threat intelligence information with the structured threat information expression (stix). Mitre Corporation 11(2012), 1–22 (2012) 2. Belcak, P., Heinrich, G., Diao, S., Fu, Y., Dong, X., Muralidharan, S., Lin, Y.C., Molchanov, P.: Small language models are the future of agentic AI (2025). https://doi.org/10.48550/arXiv.2506.02153 3. Bhagdev, R., Chapman, S., Ciravegna, F., Lanfranchi, V., Petrelli, D.: Hybrid search: Effectively combining keywords and semantic searches. In: The Semantic Web: Research and Applications (2008). https://doi.org/10.1007/978-3-540-682349_41 4. Chen, Y., Wang, W., Liu, Z.: Keyword-based search and exploration on databases. In: 2011 IEEE 27th International Conference on Data Engineering (2011). https://doi.org/10.1109/ICDE.2011.5767958 5. Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.E., Lomeli, M., Hosseini, L., Jégou, H.: The faiss library (2024). https://doi.org/10.48550/arXiv.2401.08281 6. Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., Wang, H.: Retrieval-augmented generation for large language models: A survey (2024). https://doi.org/10.48550/arXiv.2312.10997 7. Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y., Ma, S., Liu, H., Wang, S., Zhang, K., Wang, Y., Gao, W., Ni, L., Guo, J.: A survey on llm-as-a-judge (2025), https://arxiv.org/abs/2411.15594 8. Hu, Y., Zou, F., Han, J., Sun, X., Wang, Y.: Llm-tikg: Threat intelligence knowledge graph construction utilizing large language model. Computers & Security 145, 103999 (2024). https://doi.org/10.1016/j.cose.2024.103999 9. Huang, J.T., Sharma, A., Sun, S., Xia, L., Zhang, D., Pronin, P., Padmanabhan, J., Ottaviano, G., Yang, L.: Embedding-based retrieval in facebook search. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2020). https://doi.org/10.1145/3394486.3403305 10. Hutchins, E.M., Cloppert, M.J., Amin, R.M., et al.: Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains. Leading Issues in Information Warfare & Security Research 1(1), 80 (2011) 11. Jha, S., Sheyner, O., Wing, J.: Two formal analyses of attack graphs. In: Proceedings 15th IEEE Computer Security Foundations Workshop. CSFW-15 (2002). https://doi.org/10.1109/CSFW.2002.1021806 12. Kokulu, F.B., Soneji, A., Bao, T., Shoshitaishvili, Y., Zhao, Z., Doupé, A., Ahn, G.J.: Matched and mismatched socs: A qualitative study on security operations center issues. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (2019). https://doi.org/10.1145/3319535.3354239 13. Krašovec, A., Steri, G., Karopoulos, G., Trapani, M.: Large language models for cyber threat intelligence: Extracting mitre with llms. In: Availability, Reliability and Security (2025) 14. Kučečka, T., Chudâ, D., Samuhel, P.: Selective chunking — easy and effective way to estimate text similarity. In: 2013 IEEE 14th International Symposium on Computational Intelligence and Informatics (CINTI) (2013). https://doi.org/10.1109/CINTI.2013.6705226 15. Lallie, H.S., Debattista, K., Bal, J.: An empirical evaluation of the effectiveness of attack graphs and fault trees in cyber-attack perception. IEEE Transactions on Information Forensics and Security (2018). https://doi.org/10.1109/TIFS.2017.2771238
18
Deidda et al.
16. Li, Z., Xu, X., Shen, T., Xu, C., Gu, J.C., Lai, Y., Tao, C., Ma, S.: Leveraging large language models for NLG evaluation: Advances and challenges (2024). https://doi.org/10.48550/arXiv.2401.07103 17. Liu, J., Zhan, J.: Constructing knowledge graph from cyber threat intelligence using large language model. In: 2023 IEEE International Conference on Big Data (BigData) (2023). https://doi.org/10.1109/BigData59044.2023.10386611 18. Mavzer, K.B., Konieczna, E., Alves, H., Yucel, C., Chalkias, I., Mallis, D., Cetinkaya, D., Sanchez, L.A.G.: Trust and quality computation for cyber threat intelligence sharing platforms. In: 2021 IEEE International Conference on Cyber Security and Resilience (CSR) (2021). https://doi.org/10.1109/CSR51186.2021.9527975 19. Perković, G., Drobnjak, A., Botički, I.: Hallucinations in LLMs: Understanding and addressing challenges. In: 2024 47th MIPRO ICT and Electronics Convention (MIPRO). pp. 2084–2088 (2024). https://doi.org/10.1109/MIPRO60963.2024.10569238 20. Phillips, C., Swiler, L.P.: A graph-based system for network-vulnerability analysis. In: Proceedings of the 1998 workshop on New security paradigms (1998). https://doi.org/10.1145/310889.310919 21. Qiang, L., Zhengwei, J., Zeming, Y., Baoxu, L., Xin, W., Yunan, Z.: A quality evaluation method of cyber threat intelligence in user perspective. In: 2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE) (2018). https://doi.org/10.1109/TrustCom/BigDataSE.2018.00049 22. Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (2016). https://doi.org/10.18653/v1/D161264 23. Rawat, D.B., Doku, R., Garuba, M.: Cybersecurity in big data era: From securing big data to data-driven security. IEEE Transactions on Services Computing 14(6), 2055–2072 (2021). https://doi.org/10.1109/TSC.2019.2907247 24. Robertson, S., Zaragoza, H.: The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr. 3(4), 333–389 (Apr 2009). https://doi.org/10.1561/1500000019 25. Saeed, S., Suayyid, S.A., Al-Ghamdi, M.S., Al-Muhaisen, H., Almuhaideb, A.M.: A systematic literature review on cyber threat intelligence for organizational cybersecurity resilience. Sensors 23(16) (2023). https://doi.org/10.3390/s23167273 26. Sillaber, C., Sauerwein, C., Mussmann, A., Breu, R.: Data quality challenges and future research directions in threat intelligence sharing practice. In: Proceedings of the 2016 ACM on Workshop on Information Sharing and Collaborative Security. ACM (2016). https://doi.org/10.1145/2994539.2994546 27. Smith, B., Troynikov, A.: Evaluating chunking strategies for retrieval. Tech. rep., Chroma (2024), https://research.trychroma.com/evaluating-chunking 28. Su, J., Zhou, J.P., Zhang, Z., Nakov, P., Cardie, C.: Towards more robust retrievalaugmented generation: Evaluating rag under adversarial poisoning attacks (2025), https://arxiv.org/abs/2412.16708 29. Tang, F., Li, H., Zhao, M., Wu, Z., Peng, S., Yin, T.: Lrcti: A large language modelbased framework for multi-step evidence retrieval and reasoning in cyber threat intelligence credibility verification (2025), https://arxiv.org/abs/2507.11310 30. Thimmaraju, K., Rispens, S.I., Ahn, G.J.: Human performance in security operations: A survey on burnout, well-being and flow state among practitioners.
Merging CTI Through RAG and SLM for Rich Threat Representation
19
In: Proceedings 2025 Workshop on Security Operation Center Operations and Construction (2025). https://doi.org/10.14722/wosoc.2025.23002 31. Tural, B., Örpek, Z., Destan, Z.: Retrieval-augmented generation (RAG) and LLM integration. In: 2024 8th International Symposium on Innovative Approaches in Smart Technologies (ISAS) (2024). https://doi.org/10.1109/ISAS64331.2024.10845308 32. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer (2024). https://doi.org/10.1007/9783-662-69306-3 33. Xia, Y., Kim, J., Chen, Y., Ye, H., Kundu, S., Hao, C.C., Talati, N.: Understanding the performance and estimating the cost of llm fine-tuning. In: 2024 IEEE International Symposium on Workload Characterization (IISWC) (2024). https://doi.org/10.1109/IISWC63097.2024.00027 34. Xu, M., Wang, H., Liu, J., Lin, Y., Liu, C.X.Y., Lim, H.W., Dong, J.S.: IntelEX: A LLM-driven attack-level threat intelligence extraction framework (2024). https://doi.org/10.48550/arXiv.2412.10872 35. Yang, L., Wang, M., Lou, W.: An automated dynamic quality assessment method for cyber threat intelligence. Computers & Security 148, 104079 (2025). https://doi.org/10.1016/j.cose.2024.104079 36. Yucel, C., Chalkias, I., Mallis, D., Cetinkaya, D., Henriksen-Bulmer, J., Cooper, A.: Data sanitisation and redaction for cyber threat intelligence sharing platforms. In: 2021 IEEE International Conference on Cyber Security and Resilience (CSR) (2021). https://doi.org/10.1109/CSR51186.2021.9527916 37. Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: Evaluating text generation with BERT (2020). https://doi.org/10.48550/arXiv.1904.09675 38. Zhang, Y., Du, T., Ma, Y., Wang, X., Xie, Y., Yang, G., Lu, Y., Chang, E.C.: Attackg+: Boosting attack graph construction with large language models. Computers & Security 150, 104220 (2025). https://doi.org/10.1016/j.cose.2024.104220