Cost-Aware Hierarchical Multi-Agent Ransomware Detection and Family Attribution Mubashar Iqbal 1,2, Asifullah Khan 1,2,3 1
Pattern Recognition Lab, DCIS, PIEAS, Nilore, Islamabad, Pakistan.
² Deep Learning Lab, Center for Mathematical Sciences, PIEAS, Nilore, Islamabad, Pakistan. ³ PIEAS Artificial Intelligence Center (PAIC), PIEAS, Nilore, Islamabad, Pakistan. *Corresponding author(s). E-mail: [email protected]
Abstract Ransomware detection and family attribution require analysis of different modalities because it can use packing, obfuscation, process manipulation and runtime evasion techniques. However, conventional multimodal usually uses all available modalities for every sample resulting in unnecessary computational cost and increased latency. In this paper, we present a Cost Aware Hierarchical Multi-Agent System (HMAS) for adaptive ransomware detection. The proposed architecture organizes specialized agents into hierarchical domain controllers coordinated by a Meta Orchestrator. Static analysis is used as the initial low-cost modality while additional dynamic and memory modality is selectively used when confidence is insufficient or specialist agents exhibit disagreement. A cost model incorporates modality use and processing overhead. It enables the orchestration policy to balance analysis performance against computational cost. A locally deployed large language model provides verification for selected difficult cases without replacing the deterministic pipeline. Experimental evaluation compares adaptive HMAS with static only, static plus dynamic and exhaustive analysis policies across binary ransomware detection and multiclass family attribution. The complete HMAS achieved 96.57% accuracy, 0.96 F1-score and 0.99 ROC-AUC for binary detection. It also achieved 0.90 macro-F1 for family attribution. At the same time, the HMAS reduced average analysis cost by 43.97% relative to exhaustive analysis and substantially reduced average analysis latency except for the case where LLM is used. Routing analysis showed that 56.05% of cases were resolved using static evidence alone. Only 4.33% required the complete evidence pipeline. These findings demonstrate that adaptive HMAS can provide accuracy cost tradeoff for ransomware analysis while retaining support for heterogeneous and incomplete modalities.
Keywords Ransomware; Hierarchical Multi-Agent Systems; Adaptive Modality Acquisition; Static Analysis; Dynamic Analysis; Memory Forensics; Family Attribution; Cost-Aware Detection
1. Introduction
1
One of the most disruptive type of malware is rensomware due to its impact on data and services [1]. Modern ransomware uses advanced techniques like evasion, lateral movement and process manipulation to make detection and family attribution difficult [2]. Detection identifies malicious behavior and attribution links samples to specific families, which aids incident response but requires more detailed evidence [3]. Static analysis is fast and cheap, dynamic analysis reveals runtime behavior, and memory analysis uncovers volatile artifacts. Combining these provides better insights but each modality adds extra computational cost and processing time. Therefore, using all modalities for every sample is inefficient. [4].
1.1 Problem Statement The core challenge for analyzing ransomware is to balance modality sufficiency with analysis cost. An adaptive system should decide when to use more modalities based on uncertainty or contradiction [5]. Existing systems struggle with this balance especially when an individual modality is incomplete or ambiguous. The problem is formalized as choosing a modality acquisition strategy that maximizes decision reliability while minimizing cost as shown in equation 1. A* arg max[U(D, F | EA) C( EA)]
(1)
A
where C E A represents the cumulative cost of the acquired modality evidence, U D, F | E shows the utility or reliability of the resulting detection decision D and family attribution F for any given sample and controls the relative importance assigned to analysis cost. Existing ransomware detection and attribution methods are grouped by modality evidence type and processing approach but each has some limitations as pointed below.
Static analysis is fast but fails against packing, obfuscation, encryption and missing runtime behaviors [6].
Dynamic analysis reveals execution behavior but is costly, prone to latency and vulnerable to sandbox evasion [7] [8].
Memory analysis uses volatile runtime artifacts but requires extra forensic processing making it impractical for every sample [9] [10].
Fixed multimodal systems combine all evidence types but apply them indiscriminately causing unnecessary costs even when cheap static evidence is enough.
Machine learning/deep learning models optimize accuracy but do not control evidence acquisition, so they don't improve workflow efficiency [11] [12].
Agent based and multi agent systems enable task specialization and coordination [13] yet they rely on fixed workflows and do not inherently solve adaptive evidence gathering. They also struggle with uncertainty and conflicting outputs [8].
2
Overall current approaches prioritize accuracy, multimodality or specialization but fail to jointly address adaptive evidence acquisition and analysis cost leaving a gap in deciding when to acquire additional evidence before an increase in computational cost and processing latency [5].
1.2 Research Gap Existing systems lack a way to dynamically decide what modality is needed and when. Further, it should keep an account for modality acquisition cost [8]. They treat all evidence modalities as fixed inputs. They ignore the fact that acquisition of any additional modality causes delay to threat response time [4]. This calls for an adaptive evidence acquisition process instead of fixed classification. Such a process needs four main capabilities:
Estimating current confidence/uncertainty
Judging if extra evidence justifies its cost
Coordinating specialized analysis components
Setting a reliable stopping condition.
This research addresses that gap with a cost aware adaptive framework that treats static, dynamic and memory evidence as complementary but conditional sources. The goal is not just better accuracy but better accuracy cost trade off, reliable detection and attribution while reducing unnecessary expensive analysis.
1.3 Contributions The main contributions of our research are summarized below.
Cost aware hierarchical multi agent framework that integrates static, dynamic and memory analysis under unified detection and attribution logic.
Adaptive evidence acquisition dynamically escalates to costlier analysis only when current evidence is insufficient.
Cost benefit decision policy treats analysis as a sequential decision, balancing confidence gain against acquisition cost.
Hierarchical evidence coordination specialized agents feed structured findings to higher level agents for reconciled grounded attribution.
2. Background and Related Work Early ransomware detection relied on signatures and fixed rules. These were effective for known threats but failed against obfuscated variants [14]. Machine Learning techniques can now learn characteristics from samples with some static features and others dynamic [15]. Detection and family attribution are harder due to intra family variation and overlapping features. Performance depends heavily on evidence type. Some works combine multiple evidence sources, but most treat it as standard classification. They do not account for cost and efficiency of detection system [16].
3
Static analysis examines files without execution by using PE structure, imports, strings, bytes, opcodes and entropy. It is fast and cheap. It also enables rapid processing of many samples and can detect unseen families. For static analysis, features can be low level (bytes/opcodes) or high level (imports/sections) with n-grams capturing local patterns. However, packing, encryption and obfuscation can hide behavior. Thus, static analysis is best as a low cost first step whose confidence determines whether more expensive analysis is required or not [17]. Dynamic analysis executes samples in a controlled environment to record API calls, file operations, registry changes, network activity, etc., exposing runtime behavior [18]. However, dynamic analysis incurs significant time and resources overhead. This overhead depends on monitoring setup and it is vulnerable to anti VM, delayed execution and evasion. Running it for every sample wastes resources. Therefore, it should be triggered only when static evidence is insufficient. Memory forensics examines volatile artifacts (processes, loaded modules, network connections, injected code) during or after execution. It reveals activity that is invisible to static/dynamic analysis especially in case of unpacked or injected ransomware. Studies show machine learning can detect ransomware from memory data and LLMs combined with frameworks like Volatility can be used for process identification. However, memory acquisition and processing are computationally heavy and may not always be available. This evidence is valuable when earlier modalities are inconclusive. Its high cost requires selective use. Multimodal approaches combine static, dynamic and memory evidence. They use early, intermediate or late fusion to capture complementary aspects [2]. Examples include fusing PE bytes with API sequences etc. However, mostly it is assumed that all modalities are available and processed together. Static is cheap, dynamic requires a sandbox and memory is more expensive so fixed multimodal pipelines waste resources on easy cases and may lack evidence for hard ones. The real gap is not just fusion but adaptive selection of the next modality based on cost benefit [19]. In case of a ransomware attack, efficiency is increasingly important. Most evaluations ignore acquisition cost. Selective/cascaded analysis (cheap first, escalate on uncertainty) is common but relies on fixed rules. Adaptive analysis treats it as a sequential decision problem where each step has benefit and cost. The system chooses actions based on confidence, uncertainty or expected information gain [13]. Ransomware analysis fits this well because static, dynamic and memory costs differ greatly. A better framework should let the current evidence state control both the final decision and what to acquire next. This is the core direction of this research to optimize both performance and efficiency.
3. Problem Formulation and Threat Model The proposed system performs ransomware detection and family attribution while minimizing the cost of acquiring and processing security evidence from modalities. Let ( x) denote a sample and (Yd 0,1) its
4
binary label, where (1) denotes ransomware. For ransomware samples, (Y f F ) denotes the family. Given the acquired evidence EA , the system produces:
Yˆd f d ( EA ), Yˆf f f ( EA )
(2)
where ( A) defines the selected analysis strategy. The available modalities are static S , dynamic D , and memory M allowing strategies such as A S , D, M . The optimal strategy balances decision quality and analysis cost A* arg max A [U ( E A ) C ( A)]
(3)
where U EA is decision utility, C A is analysis cost, and ( ) controls the cost penalty. Thus, the objective is to minimize unnecessary evidence acquisition while maintaining reliable security decisions.
3.1 Analysis Cost Model Analysis cost is explicitly incorporated into the evidence acquisition policy. For modality i , the cost is defined as
Ci Ti Ri Ei Pi
(4)
where Ti is analysis time, Ri is computational resource consumption, Ei represents execution or analysis events and Pi represents additional processing overhead. For an analysis strategy (A)
C ( A) Ci
(5)
iA
The evaluation reports absolute cost, cost reduction and modality utilization. Cost reduction is calculated as:
CR
Cfull Cadaptive Cfull
100 (6)
This formulation enables direct evaluation of the accuracy efficiency trade-off without requiring arbitrary monetary costs. Figure 1 depicts relative cost and coverage of static, dynamic and memory modality evidence.
Figure 1: Ransomware Evidence Modalities: Relative Cost and Coverage
5
3.2 Confidence, Risk and Decision Criteria The adaptive mechanism determines whether the available evidence is sufficient for a final decision. Detection and family confidence are defined as:
Conf d max y P (Yd y | E A )
(7)
Analysis terminates when detection/family confidence exceeds the required thresholds and uncertainty remains sufficiently low.
Conf d d , Conf f f , H ( P ) H
(8)
Otherwise, additional evidence is considered. For any candidate modality, its expected utility is represented as:
EU i E[Qi ] Ci
(9)
where (E[Qi ]) is the expected improvement in decision quality. The framework therefore terminates when evidence is sufficient. It escalates when additional evidence is expected to justify its cost and rejects additional analysis when its expected benefit is insufficient. All thresholds are selected using the validation set while sensitivity analysis evaluates their effect on the accuracy cost trade-off.
3.3 Threat Model and Assumptions The system assumes ransomware adversary employing packing, obfuscation, sandbox evasion, and memory-artifact tampering to foil any single analysis modality, making static, dynamic, and memory evidence individually incomplete and motivating adaptive acquisition with hierarchical coordination. Samples are analyzed in an isolated, monitored environment. Infrastructure level attacks and LLM prompt injections are out of scope and the LLM serves only as a grounded reasoning helper not as direct evidence. Family labels are assumed reliable for supervised evaluation.
4. Proposed Cost-Aware Hierarchical Multi-Agent Framework We kept requirements from the research problem gap into consideration and propose hierarchical multiagent system for ransomware detection and family attribution. Proposed framework follows four logical levels: a Level-0 Meta-Orchestrator, Level-1 domain controllers, Level-2 specialist agents and an optional Level-1.5 LLM-based reasoning layer for difficult cases. The framework processes heterogeneous evidence from static, dynamic, network and memory modality. At the lowest level, specialist agents extract domain specific risk signals such as entropy, suspicious imports, opcode behavior, API activity, process behavior, persistence artifacts and memory indicators. Domain controllers aggregate these signals into modality level scores. The Meta-Orchestrator then applies an adaptive evidence acquisition policy. It begins with the cheapest and most broadly available evidence, typically static analysis. It escalates to additional modality only when confidence is insufficient or agent disagreement is detected. Figure 1 shows proposed framework architecture.
6
Figure 2:Complete framework architecture
The implementation uses static analysis as the preferred first stage. If the score is highly confident, the system can short circuit. Scores below 0.20 are treated as benign while scores above 0.70 is treated as sufficiently malicious to avoid unnecessary downstream execution. Ambiguous scores trigger escalation. Orchestration logic also escalates when a controller reports internal disagreement among its child agents. The Meta Orchestrator combines available modality scores using a weighted fusion rule. In the implemented baseline, static, dynamic and memory evidence are assigned fixed weights and renormalized over whichever modalities were executed. The final score is mapped to verdict bands: benign, suspicious, malicious, or ransomware. For evaluation runs, the verdict also collapsed into a binary label.
4.1 Domain Controllers Domain controllers are Level 1 agents that coordinate specialist agents within each evidence modality. Each controller receives a sample ID and features and returns an aggregate risk score, child agent outputs and an escalation flag. The Static Controller combines entropy, import and opcode/control flow analysis with weights of 0.30, 0.40, and 0.30, respectively. The Dynamic Controller combines API profiling, process monitoring, and artifact analysis with weights of 0.35, 0.40, and 0.25. The Memory Controller integrates memory forensic analysis with full modality weight. Child agents execute concurrently and excessive variance among their risk scores triggers escalation.
4.2 Specialist Agents Specialist agents are Level 2 components that produce immutable, schema validated outputs. Each output includes an agent name, sample ID, model version, risk score in [0,1], inference latency and agent-specific evidence fields. The static agents are entropy Agent, Import Profiler Agent and Opcode-CFG Agent. The dynamic agents are API Profiler Agent, Process Monitor Agent and Artifact Hunter Agent. Last is Memory Agent that reports memory risk, crypto material, C2 URLs, unpacked payload hashes and normalized
7
anomaly scores. Together, these agents provide complementary evidence for both binary ransomware detection and family attribution. Algorithm 1 depicts the entire process od proposed framework. ALGORITHM 1: HIERARCHICAL COST-AWARE DECISION ALGORITHM FOR ADAPTIVE RANSOMWARE DETECTION AND FAMILY ATTRIBUTION. Input: Sample bundle x , Available domains D, Controllers C, Optional family classifier F, LLM reviewer R Output: Final binary verdict, family verdict, confidence, audit trace 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Initialize evidence set E = empty Initialize routed domains Q = empty Select first domain d = static if available, otherwise first available domain For each selected domain d Run controller C d on evidence xd Collect child agent outputs Compute aggregate domain score sd Append d to routed domains Q Store controller output in E Compute confidence from distance of S d from decision boundary If adaptive routing is enabled If controller disagreement is detected, select next available domain Else if score is ambiguous, select next available domain Else stop evidence acquisition
sfinal weightedfusion E Fuse all acquired modality scores s final Map to deterministic baseline verdict: BENIGN, SUSPICIOUS, MALICIOUS, or RANSOMWARE If severe conflict exists and LLM reviewer R is enabled Build structured evidence prompt from controller outputs Request bounded manager decision Accept only CONFIRM, ESCALATE_ONE_TIER, or DOWNGRADE_ONE_TIER Preserve both baseline and reviewed verdicts Collapse final verdict to binary: BENIGN or RANSOMWARE If final binary verdict is RANSOMWARE Use family classifier F when available Otherwise assign generic Ransomware label Else Assign family label Benign Return final verdicts, scores, routed domains, costs, latencies and audit trace
4.3 Agent Communication and Coordination Communication among components is schema driven. Specialist agents return typed Pydantic output models, controllers return ControllerOutput and the Meta-Orchestrator returns either FusionOutput or HmasCaseOutput. These schemas preserve provenance by carrying child outputs forward instead of replacing them with only an aggregate score. Each output records risk score, latency, evidence fields and confidence related information. Controller disagreement is represented explicitly through an escalation flag. The Meta-Orchestrator records routed domains, escalated domains, conflicting domains, agent invocations, computational cost, latency and hierarchy trace. This makes each decision auditable from final verdict down
8
to individual agent evidence. When LLM reasoning is enabled, the system stores the manager prompt, manager response, debate transcript, baseline verdict, final verdict, binary labels, and family labels. Thus, the deterministic decision and any LLM-modified decision remain separately visible.
4.4 Evidence Fusion and Family Attribution The implemented baseline fusion mechanism uses a weighted score aggregation. Static, dynamic and memory scores are combined according to fixed weights. The weights are renormalized when some modalities are unavailable. This produces a final risk score in [0,1]. The final risk score is mapped into verdict bands. For family attribution, the system uses a trainable FamilyClassifier when a positive ransomware decision is made. The classifier loads trained artifacts and metadata from the family classifier directory and predicts a ransomware family from available evidence, preferring static, dynamic or memory models depending on availability. If no valid family classifier is available, the system falls back to a generic Ransomware label. Benign samples are explicitly assigned the family label Benign.
5. Implementation The proposed HMAS is implemented in Python using an asynchronous hierarchical execution model. The principal software components and their roles are summarized in Table I. Table I: Major Software Components and Their Role
Component
Technology
Primary role
Core implementation
Python, asyncio
Agent execution and concurrent task scheduling
Agent orchestration
Microsoft AutoGen
Optional LLM-agent coordination and bounded reasoning
AssistantAgent,
Single agent review and bounded Advocate Skeptic
RoundRobinGroupChat
interaction
Communication
Pydantic
Typed inter agent messages and output validation
Data processing
Pandas, NumPy
Feature representation and preprocessing
Classical ML
scikit-learn
Machine learning utilities and evaluation
Tree-based models
XGBoost, LightGBM
Specialist analytical models
Deep learning
PyTorch
Neural network based analytical components
Visualization
Matplotlib
Experimental figures and result visualization
LLM agents
The reasoning layer uses a locally hosted language model for bounded review of difficult or conflicting cases. Ransomware evidence is generated by deterministic analytical agents. Configuration settings of llm are shown in Table II. The ransomware-analysis pipeline implements adaptive evidence acquisition through the Meta-Orchestrator. It begins with low cost static analysis and escalating to dynamic and memory evidence when confidence is insufficient or evidence conflicts. Available modality scores are normalized
9
and fused to produce the final ransomware/benign decision and family attribution. The optional LLM reviewer is invoked only for selected difficult cases. Both deterministic and reviewed decisions are retained. Table II: LLM Configuration Setting
Component
Configuration
Purpose
Backend
Ollama + Autogen
Local LLM inference
Model
Phi-3 Mini, Qwen2.5-3B
Experimental reviewer
Context / output
4096 / 512 tokens
Bounded reasoning
Sampling
T=0.1, top-p=0.85, top-k=20
Controlled generation
Repetition penalty
1.05
Generation stability
Seed
42
Reproducibility
Invocation
Difficult/conflicting cases
Selective review
Input
Structured agent evidence
Evidence-grounded reasoning
Output
Bounded decision review
Verification
The executed analysis path, agent invocations, latency and cost are recorded for subsequent evaluation. The pipeline is shown in Figure 3.
Figure 3: Depiction of Ransomware Analysis Pipeline
6. Experimental Methodology The experimental evaluation investigates whether the proposed Cost-Aware Hierarchical Multi-Agent System (HMAS) can achieve reliable ransomware detection and family attribution while reducing unnecessary evidence acquisition and computational cost. The evaluation focuses on adaptive evidence routing, hierarchical coordination and bounded LLM assisted review. Four objectives are considered (i) evaluate binary detection across individual and combined evidence modalities; (ii) assess ransomware family attribution under different evidence configurations (iii) quantify the efficiency of adaptive routing using analysis frequency, latency and computational cost and (iv) determine whether bounded LLM review improves difficult or conflicting cases without replacing the deterministic analytical pipeline.
6.1 Dataset and Ransomware Families The dataset comprises ransomware, non-ransomware and benign samples with static, dynamic and memory related evidence features. Sixteen ransomware families are considered. The ransomware families contain 8013 samples and additional 8015 benign samples are available. Training data are stratified and balanced
10
during model development. No samples from the held-out families are used for training, calibration, threshold selection or hyper parameter optimization. Distribution of families in dataset is shown in Figure 4.
Figure 4: Ransomware Families Distribution in Dataset
6.1.1 Feature Extraction In our work, word features and evidence are used interchangeably. We extracted evidence from three modalities that are static, dynamic and memory. Features from each modality are normalized, encoded into latent vectors and fused into an integrated representation for detection and attribution. This enabled the framework to reason over whatever evidence is available per sample. To evaluate the effectiveness of the proposed cost aware hierarchical multi agent framework, a set of baseline systems is established for experimentations. This includes static only, dynamic only, memory only, fixed multi modal and cost aware hierarchical multi agent.
6.2 Evaluation Metrics The proposed framework is evaluated from four complementary perspectives that are detection, family attribution, decision reliability and analysis efficiency. This multidimensional evaluation is necessary because the objective of the proposed system is not simply to maximize classification performance. Binary detection performance is evaluated by treating ransomware and benign samples as the two target classes. The primary metrics are accuracy, precision, recall and F1-score. For the positive ransomware class, these measures are defined in equations 12-15. Precision
(10)
TP TP FN
(11)
PR PR
(12)
Re call F1 2*
TP TP FP
TPTN Accuracy TP FP TN FN
(13) 11
Where TP is true positive, FP is false positive, TN is true negative and FN is false negative. Because false alarms and missed ransomware samples have different operational implications, the false positive rate (FPR) and false negative rate (FNR) are also reported and defined in equation 16-17. TPR TPTP FN
(14)
FPR FPFP TN
(15)
ROC-AUC is used to evaluate discrimination across different decision thresholds. Since ransomware family attribution involves class imbalance, Precision Recall AUC (PR-AUC) is additionally reported to provide a threshold independent measure focused on the positive ransomware class. 6.2.1 Ransomware Family Attribution For ransomware positive samples, multiclass family attribution is evaluated separately from binary detection. This distinction prevents a system from receiving credit for correctly detecting ransomware when it incorrectly identifies its family. Macro-averaged F1-score is used as the primary family attribution metric. Macro F1
1 C 2.Pr ecisioni.Re calli C i 1 Pr ecisioni Re calli
(16)
6.2.2 Decision Reliability and Calibration Because the proposed framework uses confidence and uncertainty to determine whether additional evidence should be acquired, predictive confidence is evaluated in addition to classification accuracy. Expected Calibration Error (ECE) is used to measure the agreement between predicted confidence and observed correctness. Predictions are divided into confidence bins, and ECE is calculated as ECE B
b 1
nb acc b conf b N
(17)
Lower ECE indicates better calibration. Calibration is particularly important in the proposed framework because an overconfident incorrect prediction can prevent the system from acquiring additional evidence when escalation is necessary. 6.2.3 Adaptive Routing Metrics The effectiveness of the adaptive evidence acquisition mechanism is evaluated using routing specific metrics. The evidence acquisition rate for modality i is defined as shown in Equation (20). ARi Ni * 100% N
(18)
The average number of acquired modalities per sample is calculated as A
1 N Ai N i 1
(19)
Agent utilization is similarly measured using the average number of specialist agent invocations per case.
7. Results & Discussion 12
Table III presents the binary ransomware detection performance of the evaluated analysis policies. The results indicate substantial differences between the fixed evidence acquisition strategies and the proposed adaptive HMAS policy. Table III: Binary Ransomware Detection Performance
Policy
Acc.
P
R
F1
ROC-AUC
Static Only
0.9071
0.9987
0.8267
0.9046
0.9983
Static + Dynamic
0.7152
0.9216
0.5088
0.6557
0.8386
Exhaustive Analysis
0.7084
0.9667
0.4690
0.6316
0.8434
Full HMAS + LLM Review
0.9657
0.9473
0.9906
0.9685
0.9945
These results indicate that simply increasing the amount of available evidence does not necessarily improve the final decision when the evidence is combined using a fixed decision strategy. In particular, the exhaustive configuration achieved a lower recall than the static only configuration. It suggests that the evidence acquisition strategy and decision mechanism are important factors in addition to the availability of heterogeneous evidence. Complete HMAS configuration with bounded LLM review achieved an accuracy score 96.57%, 0.94 precision, 0.99 recall and 0.96 F1-score. The high recall demonstrates that the adaptive framework identified most ransomware cases while maintaining a relatively low false positive rate.
7.1 Family Attribution Performance Family attribution was evaluated separately from binary ransomware detection. Only samples identified as belonging to the ransomware class were considered for family level attribution with ransomware families treated as individual classes. This separation allows the evaluation to distinguish between the ability to detect ransomware and the ability to determine its specific family. Table IV presents the family attribution performance of the evaluated policies. The full HMAS configuration achieved the highest multiclass accuracy of 98.07%, with macro precision, macro recall and macro F1 scores of 92.71%, 0.89, and 0.90 respectively. Table IV: Family Attribution Performance
Policy
Multiclass Acc.
Macro Precision
Macro Recall
Macro F1
Static Only
0.9020
0.9778
0.6231
0.7011
Static + Dynamic
0.7329
0.9671
0.4847
0.6267
Exhaustive Analysis
0.7122
0.9657
0.4587
0.6013
Full HMAS + LLM Review
0.9807
0.9271
0.8942
0.9098
The macro-averaged measures indicate that the performance was not restricted to the highest-support families and remained strong across the evaluated family classes. Table V presents the per family results. The proposed system achieved F1-scores above 0.95 for GandCrab, LockBit, Zeppelin, WannaCry, Cerber,
13
Sodinokibi, Babuk and Akira. The highest performance was observed for WannaCry with an F1-score of 0.9936 while the lowest F1 score among the reported families was observed for Cerber at 0.9533. Table V: Per Ransomware Family Results
Family
Precision
Recall
F1-Score
Support
Benign
0.9829
0.9986
0.9907
5810
GrandCrab
0.9973
0.9874
0.9923
1902
LockBit
0.9936
0.9700
0.9817
1765
Zeppelin
0.9727
0.9468
0.9596
1392
WannaCry
1.0000
0.9874
0.9936
396
Cerber
0.9951
0.9148
0.9533
223
Sodinokibi
0.9860
0.9635
0.9746
219
Babuk
1.0000
0.9110
0.9534
191
Akira
1.0000
0.9760
0.9879
125
The relatively strong performance of lower support families is particularly relevant because macro average metrics give equal weight to each family. Nevertheless, per family support remains important when interpreting these results particularly for families such as Akira and Babuk whose samples are very less.
7.2 Cost Reduction and Efficiency Table VI compares the analysis cost and computational efficiency of the evaluated policies. Exhaustive analysis incurred an average cost of 12.0 units per case and required all available analytical domains. In contrast, the complete HMAS configuration incurred an average cost of 6.72 units per case corresponding to a relative reduction of 43.97%. Table VI: Analysis Cost and Computational Efficiency Policy
Avg. Cost
Cost Reduction
Avg. Latency ms
Agent Utilization
Static Only
4.00
Static + Dynamic
10.00
66.67
15.83
0.4286
16.67
240.11
0.8571
Exhaustive Analysis
12.00
0.00
243.72
1.000
Full HMAS
6.72
43.97
107.34
0.6231
The reduction in cost was accompanied by a reduction in average end to end latency. Exhaustive analysis required an average of 243.72 ms per case whereas HMAS required 107.34 ms representing a reduction of approximately 56%. The average agent utilization also decreased from 1.00 for exhaustive analysis to 0.6231 for HMAS.
7.3 Adaptive Routing Behavior The routing behavior of the proposed HMAS framework is summarized in Table VII. Most evaluation cases were resolved without requiring the complete evidence pipeline. Specifically, 56.05% of cases were resolved using static evidence alone 39.62% were escalated to static plus dynamic analysis and only 4.33% required the complete analysis pipeline.
14
Figure 5: Accuracy Latency Tradeoff
These results demonstrate that the proposed system does not apply an identical analysis procedure to every sample. Instead, evidence acquisition is conditioned on the intermediate state of the analysis. High confidence cases are terminated after the initial static stage whereas cases exhibiting insufficient confidence or conflicting evidence are routed to additional analysis. Table VII: Routing Behavior
Route
Percentage
Static Only
56.05
Static + Dynamic
39.62
Full Pipeline
4.33
7.4 Ablation Results The ablation experiments were conducted to determine the contribution of the principal components of the proposed HMAS architecture. Table VIII compares the complete system with configurations in which cost aware routing, adaptive escalation, hierarchical coordination, memory evidence, disagreement-based escalation and LLM assisted verification are individually removed. The comparison is performed using the same underlying deterministic decisions so that any difference can be attributed to the verification stage rather than changes in the preceding analytical pipeline. A comparison of agent’s call in case of LLM is shown in Figure 7. The ablation results indicate that the individual components contribute differently to predictive performance and computational efficiency. Removing adaptive routing is expected to increase evidence acquisition and computational cost. Further, removing additional evidence modalities primarily affects cases for which the
15
remaining evidence is insufficient. The complete system provides the reference configuration against which these changes are quantified. Figure 6 & 7 shows accuracy cost and accuracy latency tradeoff in case of HMAS and plain multi modal.
Figure 6: Accuracy Cost Tradeoff - HMAS vs Traditional Multimodal
Figure 7: LLM Role Activity Rate Table VIII: Ablation Study Policy
Avg.
Acc.
P
R
F1
ROC-A
MC Acc.
MC F1
Avg Cost
Cost Redu.
Static Only
0.9873
0.9865
0.9893
0.9893
0.9879
0.9057
0.6944
4.00
80.16
16.16
Static + Dynamic
0.7716
0.6993
0.9876
0.8188
0.8510
0.7363
0.6196
10.00
50.39
240.64
Exhaustive Analysis
0.7757
0.7038
0.9855
0.8212
0.8556
0.7148
0.5924
21.23
0.00
244.23
Full HMAS
0.9652
0.9441
0.9921
0.9675
0.9950
0.9784
0.4095
14.23
51.08
22398.88
Latency
7.5 Comparison with State of the Art 16
Comparison with previously published ransomware detection systems requires careful consideration because existing studies differ substantially in datasets, ransomware families, feature representations, execution environments, class distributions and train/test protocols. Consequently, reported performance values from different studies are not necessarily directly comparable. The primary comparison in this study is therefore conducted against the controlled baselines. Under the same dataset partitions and evaluation protocol, the proposed HMAS configuration achieved an F1-score of 0.9652 for binary ransomware detection while reducing average analysis cost by 43.97% relative to exhaustive evidence acquisition. For family attribution, HMAS achieved a macro-F1 of 0.9098 Table IX: Comparison with State of Art Techniques Method
XRan [20]
Year
2024
Evidence API, DLL, Mutex sequences
Approach
CNN + XAI
Task Ransomware detection
Reported Performance Up to 99.4 TPR Evaluated on
RansoGuard [21]
2025
Pre-Attack Sensitive APIs
RNN
Early ransomware
2390 ransomware
detection
samples across two datasets
Pulse [22]
RansomFormer [23]
Proposed HMAS
2025
2025
2026
Assembly instructions
Transformer
Zero-Day detection
PE bytes +API
Cross-modal
Ransomware
imports/calls
transformer
detection
Static + Dynamic + memory
Hierarchical Multiagent + adaptive routing
Detection + family attribution + zero day analysis
High zero day detection accuracy F199.25-99.50
96.385 binary F1; 43.97 cost reduction
7.6 Limitations Key limitations in our work include dependence on the evaluated ransomware families and datasets, reliance on evidence being obtainable, anti analysis evasion or incomplete memory acquisition, weak memory model performance, computational overhead from multi agent coordination. In case of optional LLM inference, risks from generative model inconsistency and hallucination. Further, a cost model that captures measurable computational expense but not deployment specific costs like analyst time and unresolved scalability as agent populations grow.
8. Conclusion We presented a Cost Aware Hierarchical Multi Agent System that treats ransomware detection and family attribution as a sequential decision process. It adaptively acquires evidence from static, dynamic and memory according to confidence, disagreement, availability and cost. The complete system achieved
17
96.57% binary accuracy, 0.99 recall, 0.96 F1-score and 0.90 multiclass macro-F1. It also reduced average analysis cost by 43.97%. Moreover 56.05% of samples were resolved using static evidence alone. Only 4.33% required the full pipeline demonstrating that multimodal analysis is most effective when expensive evidence is selectively acquired. The LLM was restricted to difficult or conflicting cases only. It confirmed that the primary gains resulted from adaptive routing and hierarchical coordination rather than generative reasoning.
Acknowledgements Experiments were conducted on an NVIDIA DGX system using an offline Auto Gen style stack. We thank the PR Lab, DCIS, PIEAS AI Center (PAIC), and the Center for Mathematical Sciences (CMS), PIEAS, for their support and access to computational resources.
References [1] C. Beaman, A. Barkworth, T. D. Akande, S. Hakak and M. K. Khan, "Ransomware: Recent advances, analysis, challenges and future research directions," vol. 111, p. 102490, 2021. [2] A. Alraizza and A. Algarni, "Ransomware Detection Using Machine Learning: A Survey," Big Data and Cognitive Computing, vol. 7, no. 3, p. 143, 2023. [3] L. Y. Connolly and D. S. Wall, "The rise of crypto-ransomware in a changing cybercrime landscape: Taxonomising countermeasures," Computers & Security, vol. 87, 2019. [4] Z. Liu, X. Ji and B. Peng, "A multimodal fusion method for Android malware detection based on modal independence preservation," 2026. [5] A. Ferraro, A. Galli, V. Moscato, G. M. Orlando and D. Russo, "Hierarchical multi-agent AI framework for cybersecurity in cyber-physical systems," vol. 288, p. 112556, 2026. [6] O. El Rhayati, H. Essadeq, O. El Beqqali and H. Tairi, "Structured heatmap learning for multi-family malware classification," Journal of Cybersecurity and Privacy, vol. 5, no. 3, p. 72, 2025. [7] S. Gulmez, A. Kakisim and I. Sogukpinar, "XRan: Explainable deep learning-based ransomware detection using dynamic analysis," vol. 139, p. 103703, 2024. [8] A. Ponte, D. Trizna, L. Demetrio, B. Biggio and I. T. Ogbu, "SLIFER: Investigating performance and robustness of malware detection pipelines," vol. 150, p. 104264, 2025. [9] W. Koyirar, B. Harris and J. William, "Efficient ransomware detection through process memory analysis," Techxriv Preprint, 2024. [10] Y. Wu, Y. Liu, Z. Zhang, X. Yi and L. Yi, "A swarm intelligence-enhanced ransomware detection framework via dynamic memory feature optimization," vol. 296, p. 129225, 2026. [11] I. Chaudhary and S. Adhikari, "Ransomware Detection Using Machine Learning Techniques," Researcher CAB, vol. 3, no. 1, pp. 96-111, 2024. [12] M. Cen, X. Deng, F. Jiang and R. Doss, "Zero-Ran Sniff: A zero-day ransomware early detection method based on zero-shot learning," Computers & Security, vol. 142, p. 103849, 2024. [13] A. V. Singh, E. Rathbun, E. Graham, L. Oakley, S. Boboila, P. Chin and A. Oprea, "Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defense," vol. 6, pp. 790-810, 2025.
18
[14] A. AlMajali, W. Dweik and Y. Abdelghafur, "Detecting Ransomware Through Dynamic API Call Monitoring and Machine Learning," vol. 16, no. 11, p. 5232, 2026. [15] W. F. Elsersy, A. ElShamy and M. Samy, "Ransomware detection using machine learning algorithms," in Proc. IEEE Int. Conf. Information Management and Security Applications (IMSA), 2024. [16] S. Jawad and H. M. Ahmed, "Machine Learning Approaches to Ransomware Detection: A Comprehensive Review," vol. 14, no. 6, pp. 1963-1973, 2024. [17] S. Zhang, H. Su, H. Liu and W. Yang, "MPDroid: A multimodal pre-training Android malware detection method with static and dynamic features," Computers & Security, vol. 150, p. 104262, 2025. [18] J. Carter, S. Mancoridis, P. Protopapas and B. Mitchell, "contrastBERT: Behavioral Anomaly Detection for Malware Using Contrastive Learning," in Proceedings of the Conference on Game Theory and AI for Security GameSec'25, Athens, 2025. [19] C. C. Moreira, C. C. Moreira and C. d. S. d. Sales Júnior, "A comprehensive analysis combining structural features for detection of new ransomware families," Journal of Information Security and Applications, vol. 81, p. 103716, 2024. [20] S. Gulmez and A. G. Kakisim, "XRan: Explainable deep learning-based ransomware detection," Computers & Security, vol. 139, 2024. [21] M. Cen, F. Jiang and R. Doss, "RansoGuard: A RNN-based framework," Computers & Security, vol. 150, 2025. [22] M. Gaber, M. Ahmed and H. Janicke, "Zero-day ransomware detection with Pulse," Computers & Security, vol. 148, 2025. [23] S. Alzahrani, "RansomFormer: A cross-modal transformer architecture for ransomware detection," Electronics, vol. 14, no. 7, pp. 1-20, 2025. [24] M. Egele, T. Scholte, E. Kirda and C. Kruegel, "A survey on automated dynamic malware-analysis techniques and tools," ACM Computing Surveys, vol. 44, no. 2, p. 42, 2012.
19