FUJITSU RESEARCH OF EUROPE Technical White Paper
|
·
Security Science Research Group
In collaboration with MITRE Research
|
10 June 2026
Graph-Based Structural Evaluation of LLM-Translated Adversary Emulation Procedures This technical contribution supports the MITRE white paper titled: Evaluating LLMs for Impact-Faithful Translation of Adversary Behavior Across Operating Systems Ahmed M. Elmisery Security Science Research Group
arXiv:2607.11517v1 [cs.CR] 13 Jul 2026
Fujitsu Research of Europe Limited Abstract Adversary emulation plans specify multi-step attacker procedures at the level of MITRE ATT&CK techniques, privilege requirements, and observable telemetry. Translating such plans across operating systems is necessary for cross-platform defender evaluation, and large language models (LLMs) make automated translation tractable. They also introduce a qualityassurance problem: a translation that renames tools while retaining source-platform constructs yields no usable coverage for defenders of the target platform. Binary question-based scoring tends to overestimate how faithful these translations are, because it measures countable properties rather than structural, observable, or rule-level equivalence. Graph-Based Structural Evaluation (GBSE) addresses this gap. Each procedure is modelled as a directed attributed graph, and fidelity is computed as a normalized Graph Edit Distance (GED) across four progressively stricter node-matching layers: technique (L0), tactic (L1), telemetry class (L2), and Sigma logsource (L3). Applied to the full 29-step ALPHV/BlackCat Windows→Linux plan, with a genuine native-Windows control reconstructed step-for-step from the conversion record and a Linux variant taken unmodified from the LLM output, the framework finds: technique and tactic structure is preserved across the OS boundary (GED=0, Simstruct =1.000); telemetry fidelity drops to Simstruct =0.897 (GED=3), driven by three steps that emit an unmapped observable class or drift in telemetry; and independent Sigma-layer matching recovers to Simstruct =1.000 across all 29 steps. Every state classifies as Medium Fidelity (best composite S=0.674), and the deployment gate (S≥0.80, requiring technical realism ≥ 0.990 against the measured 0.43) is unreachable at current evaluation quality. The layer scores are numerically identical to those obtained under an earlier same-procedure design, which confirms that the layers decompose cleanly along the OS-abstraction axis. The framework includes a bipartite-GED implementation, a telemetry-intent parser that derives structured observable classes from free-text annotations, and a validated library of 49 Sigma detection rules (19 Linux, 30 Windows) that gives complete ATT&CK technique coverage of the procedure and passes the Sigma specification validator with zero findings. A supplementary analysis recovers a genuine technique-level divergence (for example RDP-based external access reassigned to unencrypted exfiltration, and credential-store access reassigned to remote-system discovery) that a procedure-aligned view necessarily suppresses. All numerical results in this paper were reproduced from the reference implementation and asserted against the recorded pipeline outputs.
Keywords: adversary emulation, MITRE ATT&CK, graph edit distance, large language models, cross-platform translation, Sigma, detection engineering, CALDERA.
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
Notation Symbol
Definition
G = (V, E, φV , φE )
Directed attributed procedure graph: step nodes, dependency edges, node/edge attribute maps. Control graph (human-validated source) and variant graph (LLM translation). Minimum-cost edit path transforming Gc into Gv . 1 − GED/ max(|Vc |, |Vv |). Trial normaliser max(29, 29) = 29. |A ∩ B|/ max(|A|, |B|), the Szymkiewicz–Simpson overlap coefficient. L1 tactic and L3 Sigma thresholds (inclusive, ≥). L2 telemetry threshold (strict, >; boundary 0.50 fails). Behavior Chain Fidelity = 0.5 · auto + 0.5 · Simstruct . Composite score = 0.4 · BCF + 0.3 · TR + 0.3 · DV. Technical Realism (0.43) and Defensive Value (0.51) from manual evaluation. Off-diagonal sentinel in the bipartite cost matrix.
Gc , G v GED(Gc , Gv ) Simstruct (Gc , Gv ) szss(A, B) θtac = θsig = 0.50 θtele = 0.50 BCF S TR, DV INF = 109
2
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
1 Introduction and Motivation An adversary emulation plan is an ordered description of an attacker’s procedure, written so that a defender can replay it under controlled conditions. Each step names an ATT&CK technique, states the privilege it requires, and indicates the telemetry a defender should expect to observe. Plans are authored for a particular operating system because the concrete tradecraft is OS-specific: a Windows plan uses mstsc.exe, bitsadmin, and registry edits, whereas the same adversary objective on Linux uses ssh, wget, and sysctl. Defenders who run mixed estates need the same procedure expressed on both platforms, and producing the second version by hand is slow. Large language models can translate a plan from one operating system to another in seconds. The difficulty is that an LLM optimizes for plausible-looking output, not for defender-relevant equivalence. A model will happily rename a Windows tool to a Linux one while keeping the original command’s observable footprint, or worse, keep the Windows binary itself and wrap it in a compatibility layer. The translated step then looks correct at the level of technique labels but generates the wrong evidence on the target host, so detection content written for that platform never fires. The question this paper answers is how to measure that gap precisely enough to route a translation to deployment, to manual review, or to regeneration. The starting point is a measurement limitation in the question-based evaluation that GBSE extends. The baseline framework scores a translation along three axes, Behavior Chain Fidelity (BCF), Technical Realism (TR), and Defensive Value (DV), using automated checks for sequence, ATT&CK alignment, privilege progression, and telemetry presence. The baseline study itself cautions that a high automated pass rate must not be read as a successful translation for operational emulation, because the checks are static counts and do not validate structural, semantic, or runtime equivalence. The two observations below make that caution concrete. Empirical Observation 1 (The measurement discrepancy). Binary evaluation instruments systematically overestimate cross-OS translation fidelity. A translation can record a high automated pass rate while, under telemetry-aware structural analysis, exhibiting low defender-relevant fidelity. Binary scoring masks at least four failure modes: step insertion (a technique-valid step that shifts the expected telemetry profile and tactical flow); edge reordering (a dependency change that leaves the step count intact but renders sequence-correlation rules inert); telemetry class drift (a tool substitution that drops an observable class, for example replacing process telemetry with purely network telemetry); and Sigma logsource gap (a step that keeps its technique identifier while shifting its logsource to a platform where no equivalent native rule exists). Empirical Observation 2 (Concrete instance). In the ALPHV/BlackCat trial a lateral-movement step illustrates the discrepancy directly: a Windows PsExec-style action that produces both process and network observables is rendered on Linux as an scp/ssh loop dominated by network observables. Process-based lateral-movement detection becomes ineffective even though the high-level technique correspondence appears preserved. These failures are invisible to a flat list of steps but explicit once the procedure is treated as a graph. The remainder of this paper develops that treatment, applies it to the full 29-step ALPHV/BlackCat plan, and reports a verified set of layer and composite scores together with the detection content the analysis produced.
2 Background and Assumptions
3
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
2.1 ATT&CK, emulation, and the detection stack ATT&CK organizes adversary behavior into tactics (the attacker’s goal at a stage) and techniques (the method used to achieve it), each carrying a stable identifier such as T1021.001. CALDERA executes emulation plans expressed in this vocabulary. Sigma is a portable signature format whose rules bind a logsource (a product and an event category, for example linux/process_creation) to a detection condition over event fields; rules compile to native queries for Elastic, Splunk, QRadar, and Microsoft Sentinel. D3FEND catalogues defensive countermeasures and the observables they consume. The four layers of GBSE read attributes drawn from exactly these sources, which is what lets a single similarity score carry a defender-relevant interpretation rather than a purely topological one. 2.2 Assumptions The framework rests on three assumptions that the evaluation later tests rather than presumes. A1 (OS-independence of labels). ATT&CK technique and tactic identifiers are deliberately platform-neutral. A faithful translation preserves them across an OS boundary, so agreement at L0/L1 is a genuine measurement of technique preservation, not an artefact of comparing a procedure to itself. A2 (OS-agnostic observable intent). The telemetry a step is expected to produce can be expressed independently of the host: a step that touches the filesystem, opens a socket, and spawns a process produces {file, network, process} regardless of whether it runs wget or bitsadmin. Telemetry drift is therefore a property of the translation, not of how the source command happens to be spelled. A3 (OS-specificity of detection). A Sigma rule’s logsource is platform-bound by construction. Detection equivalence across an OS boundary holds only when both sides reduce to a common event category, which for command-line tradecraft is process_creation. A1 and A2 predict that a faithful cross-OS translation will agree at L0/L1 and at L2 except where the variant genuinely drifts; A3 predicts that detectability must be measured at the logsourcecategory level. The results in Section 5 are consistent with all three, and Section 5.7 shows that the layer scores are invariant to whether the control is expressed in source-OS commands, which is the sharpest available test of A1–A3 holding simultaneously. 2.3 Scope This paper develops the structural-evaluation of the methodology. The node attributes consumed by the matching layers are assumed to be present in the enriched procedure graphs; the upstream semantic-enrichment pipeline that produces them is treated as a black box that emits attributed steps. The evaluation is single-procedure (ALPHV/BlackCat) and single-pair (one control, one variant); the framework is general, but the empirical claims are stated for this trial only.
3 The GBSE Framework 3.1 Procedure graph formalism Each control and variant procedure is represented as a directed attributed graph. This captures step-level semantics, dependency order, and global topology at once, and gives a formal substrate for asking whether a translated procedure instantiates the same adversary chain rather than merely reusing the same technique labels. Definition 1 (Procedure graph). A procedure is a directed attributed graph G = (V, E, φV , φE ), 4
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
where V is the set of procedural-step nodes and E ⊆ V × V is the set of directed dependencies. The map φV assigns each node its attributes, φV (v) = htechnique_id, tactic[], privilege_context, telemetry_classes[], sigma_rules[], . . . i, and the map φE assigns each edge its type: sequential precedence (w=1.0), conditional-privilege dependency (w=1.5), or data-flow / telemetry-trigger relation (w=0.5). In the trial both Gc (Windows) and Gv (Linux) carry 29 nodes and 28 sequential edges. Because a faithful translation preserves the per-step technique and tactic, the step-aligned technique identifiers match even though the underlying commands change from Windows to Linux. The discriminating signal therefore lives below the label layer, at telemetry (L2), Sigma logsource (L3), and the out-of-band os_constructs check. 3.2 Structural similarity via normalized GED Fidelity is defined by how much typed editing is needed to turn the control graph into the variant graph. The edit operations are chosen so that each carries a procedural meaning. Proposition 1 (Structural similarity as a continuous fidelity measure). Let Gc be the control graph and Gv the variant graph. If procedure fidelity is the amount of semantic and topological editing required to transform Gc into Gv , then Simstruct (Gc , Gv ) = 1 −
GED(Gc , Gv ) max(|Vc |, |Vv |)
(1)
is a continuous measure of structural correspondence on [0, 1]. When node and edge edits are semantically typed, the score decomposes into procedurally meaningful deviations: node insertion = an added operational prerequisite, node substitution = altered execution or observability semantics, and edge substitution = changed dependency logic. The score partitions into three fidelity regions that act as routing signals: above 0.80 is high fidelity, 0.60 to 0.80 is medium, and below 0.60 is low. These bounds are decision thresholds intended for empirical calibration rather than fixed constants. 3.3 Layered semantic enrichment The node-matching relation is refined in three steps. Each layer subsumes the previous one and adds exactly one defender-relevant constraint, so the layer at which similarity first drops localizes the kind of deviation. L0 (Technique). Nodes match iff technique_id is equal. This is the ontology layer. L1 (Tactic). L0 holds and the tactic sets overlap, szss(T1 , T2 ) ≥ 0.50. A drop here means strategic drift: an inserted execution-oriented step inside a privilege-escalation chain can stay locally plausible while changing the tactical role of that segment. L2 (Telemetry). L1 holds and the telemetry classes overlap strictly, szss(C1 , C2 ) > 0.50. A drop here means observational drift. This is the critical layer for the trial: Windows-to-Linux conversion that approximates a Windows PE through a compatibility layer can replace native Windows API effects with Linux process or identity telemetry, which binary scoring misses but telemetry-aware comparison exposes. The threshold is strict, so a boundary overlap of exactly 0.50 fails. 5
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
L3 (Sigma logsource). L2 holds and the Sigma logsource categories overlap, szss(L1 , L2 ) ≥ 0.50. A drop here means detectability drift: a step that stays ATT&CK-aligned but no longer maps to an equivalent logsource-bearing detection is, from a defender’s view, degraded. Heuristic 1 (Progressive enrichment). The four layers are not four independent scorers but four increasingly strict versions of one node-matching relation, adding in order: identity, strategic role, observability profile, and rule-bearing detectability. This makes the location of any fidelity loss interpretable rather than opaque. A separate independent L3 relation (L3-i) drops the L2 prerequisite and requires only L0 plus logsource overlap. Section 5.6 shows why this matters: chained L3 makes detection coverage invisible whenever L2 has already failed, whereas independent L3 measures detectability on its own terms. 3.4 Algorithm 1: telemetry-intent parser For L2 to be a fair cross-platform comparison, the control’s telemetry must be expressed in the same vocabulary as the variant’s. Control steps carry free-text telemetry_expected annotations; Algorithm 1 derives structured telemetry_classes from them using an OS-agnostic keyword map over the four standard classes {file, network, process, identity}. This enrichment is what lets the three telemetry failures in Section 5.3 be attributed to the variant rather than to a gap in the control annotations. Algorithm 1: parse_telemetry_expected — structured observable classes from text Input: free-text annotation t; keyword map K : class 7→ keyword set Output: sorted list of observable classes 1 t ← lower(t) 2 out ← ∅ 3 foreach (cls, kws) ∈ K do 4 if ∃ k ∈ kws such that k ⊆ t then 5 out ← out ∪ {cls} 6 end 7 end 8 return sorted(out)
3.5 Algorithm 2: bipartite GED Exact GED is NP-hard, so the framework uses the Riesen–Bunke bipartite approximation: build a cost matrix over node substitutions, insertions, and deletions, then solve the assignment with the Hungarian algorithm in O(n3 ). The matching predicate is the layer relation of Section 3.3, so the same GED routine computes every layer’s score by swapping the predicate. ▶ Implementation fix. In earlier implementations the off-diagonal entries of the (nc +nv )×(nc +nv ) cost matrix were initialized to 0 instead of to a sentinel. This admitted phantom zero-cost assignments and corrupted every layer score. The corrected routine initializes the matrix to INF = 109 and writes only the legal substitution, deletion, insertion, and null-to-null costs. All scores in this paper are from the corrected routine.
6
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
Algorithm 2: bipartite_ged( C-steps, V -steps, match) 1 nc ← |C|, nv ← |V |
2 C ← INF · 1(nc +nv )×(nc +nv )
// sentinel init (the fix)
3 for i ∈ [0, nc ), j ∈ [0, nv ) do 4
C[i][j] ← 0 if match(Ci , Vj ) else 1
// substitution
5 end 6 for i ∈ [0, nc ) do 7
C[i][nv +i] ← 1
// deletion (diagonal only)
8 end 9 for j ∈ [0, nv ) do 10
C[nc +j][j] ← 1
// insertion (diagonal only)
11 end 12 for i ∈ [0, nv ), j ∈ [0, nc ) do
C[nc +i][nv +j] ← 0 14 end 15 (row , col ) ← hungarian(C) ∑ 16 GED ← C[row , col ( ] ) 17 return GED, max 0, 1 − GED/ max(nc , nv ) 13
// null-to-null
The layer predicates stack as follows. match at L0 tests technique equality; L1 adds the tactic overlap; L2 adds the strict telemetry overlap; chained L3 adds the logsource overlap on top of L2; independent L3 requires only L0 and the logsource overlap. 3.6 Algorithm 3: Sigma injection Before L3 can be scored, each node must carry the Sigma rules that apply to it. Algorithm 3 attaches a rule to a step when the rule’s ATT&CK tag matches the step’s technique or when the rule’s telemetry overlaps the step’s by at least the threshold. The logsource category used for L3 is the final path segment of the rule’s logsource, which is process_creation for command-line tradecraft on both platforms. Algorithm 3: inject_sigma( step v, library) 1 out ← ∅
2 foreach rule r ∈ library do
tm ← (v.technique_id ∈ r.attack_tags) 4 ov ← szss(v.telemetry_classes, r.telemetry) 5 if tm or ov ≥ 0.50 then 6 attach r to out with logsource_category = lastsegment(r.logsource) 7 end 8 end 9 return out 3
3.7 Composite scoring and the deployment gate The structural score feeds a composite that keeps adversary-intent preservation primary while treating technical realism and defensive value as secondary dimensions of equal weight: BCF = 0.5 · auto_pass + 0.5 · Simstruct ,
S = 0.4 · BCF + 0.3 · TR + 0.3 · DV.
(2)
A score above 0.80 is High Fidelity and clears the CALDERA deployment gate; 0.60 to 0.80 is Medium Fidelity; below 0.60 is Low Fidelity and routes into a tool-interactive critique loop. The weights are initial analytic settings pending empirical calibration, not an experimentally validated 7
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
scoring law. They embed Equation (1) into BCF so that structural fidelity and the automated pass rate contribute equally to the behavior-chain term.
4 Experimental Setup and Workflow 4.1 The genuine cross-OS design The evaluation compares a native-Windows control against the LLM’s Linux output. Both graphs are drawn from the same pipeline run. The Linux variant Gv is the full_output stage of the pipeline, used unmodified. The Windows control Gc is reconstructed step-for-step from the pipeline’s own conversion record, so that each control step carries the genuine Windows command on which the documented Windows-to-Linux translation acted. Every reconstructed command is grounded in recorded evidence, and the grounding source is stored per step in a win_command_provenance field. Table 1 gives the provenance taxonomy and its distribution over the 29 steps. Table 1. Provenance of the reconstructed Windows control commands. Each of the 29 control steps is grounded in one recorded source, in strict priority order. Tag
Priority
conversion_note
1
13
cross_platform
4
8
bitsadmin_map
3
4
de_wine
2
2
technique_pattern
2b
2
Total
Steps
Grounding Exact or prefix match against conversion_notes[].original_action (direct ground truth). Dual-platform binary (rclone, scp, native PE) retained with the Linux-only sudo prefix stripped. wget download of a PE mapped to the documented bitsadmin /transfer form. Provable inverse of a wine-wrapped PE: wine <pe> <args> → <pe> <args>. Generalisation of the two documented mstsc↔ssh and netsh↔systemctl conversion patterns to residual steps of the same technique.
29
Because A1 holds, agreement at L0/L1 is now a genuine measurement that the LLM preserved the technique and tactic sequence across the OS boundary, not a procedure compared to itself. The os_constructs check is likewise a real defect measure: nine wine/.exe Windows constructs survive into the Linux Gv at steps {8, 9, 11, 12, 18, 20, 21, 26, 27}, scored against a true Windows baseline. 4.2 The ALPHV/BlackCat procedure The procedure is the 29-step ALPHV/BlackCat Windows-to-Linux emulation plan. Its technique spine, shared by both graphs at L0, is T1048.003, T1105, T1021.001×3, T1087.002, T1087.001, T1018, T1003.001×3, T1562.001×2, T1112, T1046, T1077, T1059.001×2, T1021.004, T1059.004, T1083, and T1048.001, covering 16 unique techniques. Table 2 shows representative steps with the control command and the telemetry classes on each side. 4.3 The Sigma rule library Detection content is supplied by a single validated library, sigma_rules_gbse.yml, comprising 49 rules: 19 Linux rules (L01–L19, injected into Gv ) and 30 Windows rules (W01–W30, injected into 8
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
Table 2. Representative steps: reconstructed Windows control command and the telemetry classes of Gc (Algorithm 1 parsed) versus Gv . The trio S13/S14/S27 is the source of the L2 loss; all other steps overlap at ≥ 0.667. Step
Technique
Control (abridged)
command
S1
T1048.003
S2
T1105
S3
T1021.001
S13
T1562.001
S14
T1562.001
S27
T1059.001
rclone serve webdav --addr :8080 wget http://c2.host/... -O /tmp/cs ssh [email protected] "bash -l" netsh advfirewall set ... state off netsh advfirewall set ... state off /tmp/collector1.exe /remote ...
Gc telemetry
Gv telemetry
file, network, process
file, network, process
file, network, process
file, network, process
identity, network, process
identity, network, process
file, process
other, process
file
other, process
file, network, process
identity, process
Gc ). Every rule targets the process_creation logsource category and is derived from an actual ALPHV/BlackCat command. The library gives complete ATT&CK technique coverage of the procedure on both platforms, so under Algorithm 3 every step receives at least one rule on each side, and the shared process_creation category yields logsource overlap 1.0 for all 29 steps under independent L3. A validation pass corrected four rule-condition defects (an undefined selection identifier in W02; bare selection blocks in L11, W05, and W13 orphaned by a “1 of selection_*” condition) and added eleven Windows rules W20–W30 to close per-step firing gaps, including W29 (T1133) and W30 (T1555) for the gold-control divergence of Section 5.8. The present paper verified the full 49-rule library against the Sigma specification with zero findings; the procedure for that verification is given in Appendix C. 4.4 Pipeline workflow The reference build executes the following stages, each of which maps to a component above. Figure-free, the workflow is: 1. Load the OS-agnostic step spine, the Linux variant, the conversion notes, the evaluation result, and the OS-violation list from the pipeline output. 2. Reconstruct Gc : derive each Windows command via the priority order of Table 1, set os=Windows, parse telemetry_expected into structured classes with Algorithm 1, map privilege, and inject the Windows rules. 3. Build Gv : normalize the Linux variant’s telemetry, privilege, and tactic fields, and inject the Linux rules. 4. Automated evaluation: run the twelve checks (Section 5.1) including the os_constructs defect count against the Windows baseline. 5. Layer evaluation: run Algorithm 2 once per layer (L0, L1, L2, L3 pre-sigma, L3 chained, L3 independent), recording GED, similarity, and the failing pairs with a per-failure diagnosis. 6. Composite scoring: combine the automated pass rate, the structural similarity, and the manual TR/DV via Equation (2), and test the deployment gate. 9
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
7. Gold-control divergence: recover the 7-step technique substitutions from the recorded evaluation result (Section 5.8). 8. Persist the enriched control graph, the enriched variant graph, and the full results document as JSON. A human reviewer sits at stages 4 through 7: the framework computes and routes, but a translation is promoted to deployment, sent to manual review, or returned for regeneration by a person reading the layer diagnostics, not automatically. The intended mode is supervisory, not autonomous.
5 Results All figures in this section are from the 29-step run, normaliser 29 throughout, and were reproduced from the reference implementation as recorded in Appendix C. 5.1 Automated evaluation Nine of twelve automated checks pass, giving an automated pass rate of 0.7500. The three failures are: os_constructs_valid (nine steps of the Linux variant still invoke wine for Windows .exe execution, a translation defect against the Windows baseline); extra_telemetry_classes (the variant emits an other class outside the standard vocabulary {file, network, process, identity}); and privilege_progression_match (six of 29 steps mismatch on privilege). All technique-identifier, tactic, diversity, and multi-class checks pass. The pass rate feeds BCF. 5.2 Layered structural results Table 3 reports GED, similarity, and BCF at each layer. Technique and tactic structure is preserved perfectly; telemetry costs three edits; chained L3 cannot recover because it requires L2 to pass first; independent L3 recovers fully. Table 3. GBSE layer results (29-step, normaliser 29). Green rows are Simstruct =1.000; amber rows are Simstruct =0.897. Layer
GED
Sim
BCF
Signal
Baseline (L0) L1 + tactic L2 + telemetry
0.0 0.0 3.0
1.000 1.000 0.897
0.875 0.875 0.823
L3 pre-sigma L3 post (chained)
3.0 3.0
0.897 0.897
0.823 0.823
L3 post (independent)
0.0
1.000
0.875
Perfect technique preservation; all 29 identifiers match. All 29 tactic fields match step-for-step. 3 of 29 fail. S13 szss=0.50 fails strict; S14 szss=0.00; S27 szss=0.333. Root cause: variant other class. Identical to L2; no Sigma rules in source files. S13/S14/S27 still fail; chained formulation requires L2 first, so injection cannot bypass the telemetry gate. All three failures resolved. S13/S14 via W09/L09; S27 via W15/L15; logsource overlap 1.0 everywhere.
5.3 L2 failure root cause The three L2 failures originate entirely on the variant side. At S13 and S14 (T1562.001) the variant emits an unmapped other telemetry class, and at S27 (T1059.001) the variant substitutes {identity, process} for the control’s {file, network, process} while the tactic stays unchanged. Because the control telemetry is fully structured by Algorithm 1 in this run, the failures reflect variant output quality rather than gaps in the gold annotation. 10
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
Table 4. L2 analysis. Three of 29 steps fail; all are variant-side telemetry issues. Step
Technique
Telemetry (control → variant, overlap)
L2
S13 S14 S27 S2 S10 S†
T1562.001 T1562.001 T1059.001 T1105 T1021.001 (24 others)
[file, process] → [other, process], szss=0.50 [file] → [other, process], szss=0.00 [file, net, proc] → [identity, proc], szss=0.333 [file, net, proc] → [file, net, proc], szss=1.00 [id, net, proc] → [id, net, proc], szss=1.00 various, all szss ≥ 0.667
FAIL FAIL FAIL PASS PASS PASS
5.4 Composite scores and the deployment gate Table 5 applies Equation (2). Every state classifies as Medium Fidelity. Independent L3 is the best state at S = 0.674, lifted by a higher Defensive Value (0.65) because full detection coverage is realized. Table 5. Composite scores (29-step run, automated pass rate 0.75). All states are Medium Fidelity. State
Sim
BCF
TR/DV
S
Classification
Baseline / L1 L2 telemetry L3 pre-sigma L3 post (chained) L3 post (indep.)
1.000 0.897 0.897 0.897 1.000
0.875 0.823 0.823 0.823 0.875
0.43 / 0.51 0.43 / 0.51 0.43 / 0.51 0.43 / 0.51 0.43 / 0.65
0.632 0.611 0.611 0.611 0.674
Medium Fidelity Medium Fidelity Medium Fidelity Medium Fidelity Medium Fidelity
The deployment gate S ≥ 0.80 is unreachable at current evaluation quality. With the best BCF = 0.875 and DV = 0.51, clearing the gate requires TR = (0.80 − 0.4 · 0.875 − 0.3 · 0.51)/0.30 = 0.990, against a measured TR = 0.43. The best achievable composite is therefore 0.674. 5.5 Sigma injection: chained versus independent The distinction between the two L3 relations is operationally decisive. Under chained L3, the three steps that already failed L2 stay failed regardless of detection coverage, because the chained predicate evaluates the telemetry gate first. Under independent L3, all three are resolved through technique-tag matches: S13 and S14 attach W09/L09 (T1562.001) and S27 attaches W15/L15 (T1059.001), each giving logsource-category overlap 1.0. All 29 steps pass independent L3, and the composite rises from 0.611 to 0.674, a relative improvement of about 6.2%. 5.6 Score invariance under the cross-OS design Every layer score and every composite score above is numerically identical to the values obtained under an earlier design in which both graphs were drawn from the same Linux procedure at two pipeline stages. The automated battery is unchanged at 9/12, and the gate stays unreachable. This invariance is a validity check on the layered construction, not a coincidence. The reason follows from assumptions A1–A3. L0/L1 read OS-independent labels, so a faithful translation preserves them whether the control is spelled in Windows or Linux commands. L2 reads telemetry classes derived from OS-agnostic intent, so the three telemetry-drift failures are intrinsic to the variant. Independent L3 reduces every step to a process_creation category common to the Windows and Linux rule families, giving overlap 1.0 at both ends. The discriminating quantities never depended on the control’s OS surface. What changes under the genuine design 11
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
is interpretability, not arithmetic: GED=0 at L0/L1 is now a real cross-OS preservation measurement, and os_constructs_valid failing on nine surviving wine/.exe constructs is now a real defect score against a Windows baseline. 5.7 Gold-control technique divergence The 29-step reconstruction aligns each control step to its documented Windows pre-image, so techniques agree by construction and L0 reports zero mismatches. That is correct for measuring procedure-level fidelity, but it suppresses a coarser signal that the pipeline recorded separately. The pipeline’s evaluation result references a 7-step Windows gold control scored against the 29step Linux variant. Recovering that residue surfaces a genuine technique-level divergence at six of the seven gold steps, shown in Table 6. Table 6. Technique-level divergence between the 7-step Windows gold control and the 29-step Linux variant. Six of seven gold steps are reassigned to a different ATT&CK technique. Gold step
Windows gold technique
Linux variant technique
1
T1133 External Remote Services / RDP
3 4 5 6 7
T1087.002 Account Discovery: Domain T1105 Ingress Tool Transfer T1021.001 Remote Services: RDP T1105 Ingress Tool Transfer T1555 Credentials from Stores
T1048.003 Exfiltration over unencrypted non-C2 T1021.001 Remote Services: RDP T1021.001 Remote Services: RDP T1087.002 Account Discovery: Domain T1087.001 Account Discovery: Local T1018 Remote System Discovery
Two substitutions are operationally significant. At gold step 1 the Windows external-remoteservices vector T1133 (RDP exposure) maps to T1048.003 (exfiltration over an unencrypted non-C2 protocol), and at gold step 7 the Windows credential-store access T1555 maps to T1018 (remotesystem discovery). These are technique reassignments, not faithful preservations, and a procedurealigned view cannot expose them. At the granularity the pipeline’s own gold control encodes, the Windows-to-Linux translation does not preserve all ATT&CK techniques. The two coverage rules W29 (T1133) and W30 (T1555) are motivated precisely by these two substitutions and are present in the validated library.
6 Discussion 6.1 What the layers reveal Evaluation sensitivity changes materially as semantic layers are added. Under technique-only matching the translation looks acceptable; under tactic-aware matching no strategic drift appears; under telemetry-aware matching the procedure loses three edits to observational drift; and under Sigma-aware matching with adequate target-platform content, detectability is fully restored. The practical lesson is that defender-relevant evaluation is layered. What is faithful at the ontology layer can still be unfaithful at the telemetry and rule-coverage layers, and a single similarity number hides that unless the layer at which it was computed is stated alongside it. The trial also separates two error classes that binary scoring conflates. The L2 failures are outputquality errors that a better-conditioned translation would avoid (an unmapped telemetry class, a drift at one step). The os_constructs failures are translation-completeness errors: nine steps were not translated at all but wrapped in a compatibility layer, so they cannot produce native Linux telemetry no matter how the rest of the pipeline is tuned. The first class is fixable downstream; the second requires re-translation. 12
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
6.2 Guidance drawn from the analysis The following five items follow directly from the results and are stated as remediation guidance, with the layer each one moves. G1. Remove the other telemetry class from variant output. The variant emits an unmapped other class at S13 and S14, which fails the strict L2 threshold against the control’s {file, process}. Post-processing that maps other to a concrete class via D3FEND resolves two of the three L2 failures. G2. Re-translate S27 with native tradecraft. At S27 the variant drifts to {identity, process}. Re-running the step with a native Linux interpreter for T1059.001 restores telemetry that matches the source and closes the third L2 failure. G3. Use independent L3 as the primary detectability metric. Chained L3 hides detection coverage whenever L2 has failed. Independent L3 measures detectability on its own terms, is more reliable in sparse-control settings, and reaches Simstruct =1.000 on all 29 steps, raising the composite from 0.611 to 0.674. G4. Re-run the trial with a native Linux translation. Because the deployed variant retains wine/.exe at nine steps, the four-layer evaluation cannot yet exercise genuine technique and telemetry differences for those steps. A native re-translation would make the full evaluation meaningful and would likely surface deviations the compatibility layer currently masks. G5. Recalibrate the gate and the manual instrument. The gate S ≥ 0.80 demands TR ≥ 0.990, which is unachievable at the present TR = 0.43. A two-tier gate, with S ≥ 0.60 for conditional deployment (achievable now) and S ≥ 0.70 for full deployment, is more useful. 6.3 Limitations Three limitations bound the claims. First, the evaluation covers one procedure and one control– variant pair; the framework generalizes, but the numbers do not. Second, the composite weights and the fidelity-band thresholds are analytic settings awaiting empirical calibration, so the composite scores should be read as relative rankings between states, not as absolute fidelity. Third, the manual TR and DV inputs rest on a marginal-agreement instrument; until the questions are respecified, the composite carries that uncertainty. None of these affects the layer GED scores, which are deterministic functions of the enriched graphs and were reproduced exactly.
7 Related Work Graph Edit Distance has a long history as a structural-similarity measure, and the bipartite approximation used here is the Riesen–Bunke assignment formulation [2, 3], which trades exactness for an O(n3 ) solution via the Hungarian algorithm. The evaluation vocabulary is ATT&CK [1], the detection format is Sigma [6], the countermeasure ontology is D3FEND [7], and the execution target is CALDERA [8]. The procedure under study is the ALPHV/BlackCat ransomware tradecraft documented in the joint advisory [9], and the translation pipeline that produced the variant is the LLM-based OS-translation pipeline [10]. The composite-scoring philosophy, and the routing of low-fidelity translations into a tool-interactive critique loop, draws on self-correction work for LLMs [4, 11]. Inter-rater reliability is reported with Fleiss’ κ [5], and the overlap coefficient at the heart of every layer predicate is the Szymkiewicz–Simpson coefficient [12].
8 Conclusion Treating an adversary emulation procedure as a directed attributed graph turns a vague question, “is this translation faithful?”, into a layered measurement that says where fidelity is lost.
13
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
Applied to the full ALPHV/BlackCat Windows-to-Linux plan against a genuine native-Windows control, the framework finds perfect technique and tactic preservation, a three-edit telemetry loss attributable to the variant, and full detectability recovery under independent Sigma-layer matching. Every state is Medium Fidelity, and the deployment gate is unreachable at current evaluation quality, which is itself a useful and honest result: it says the translation is not yet ready for unattended emulation and identifies the two specific repairs and the gate recalibration that would change that. The score invariance under the cross-OS design confirms that the layers decompose cleanly along the OS-abstraction axis, and the validated 49-rule library makes the detectability layer concrete and deployable. The framework is a measurement instrument and a routing aid, with a human reading the diagnostics at every decision point.
References [1] MITRE Corporation. ATT&CK Enterprise Framework v15. 2024. https://attack.mitre.org/. [2] K. Riesen and H. Bunke. Approximate graph edit distance computation by means of bipartite graph matching. Image and Vision Computing, 27(7), 2009. [3] Z. Zeng et al. Comparing Stars: On Approximating Graph Edit Distance. PVLDB, 2:25–36, 2009. [4] Z. Gou et al. CRITIC: LLMs Can Self-Correct with Tool-Interactive Critiquing. ICLR, 2024. [5] J. L. Fleiss. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 1971. [6] T. Roth and T. Patzke. Sigma: Generic Signature Format for SIEM Systems. BSides Munich, 2017. [7] MITRE Corporation. D3FEND: A Knowledge Graph of Cybersecurity Countermeasures. 2021. [8] MITRE Corporation. CALDERA: Cyber Adversary Language and Operations, v4. 2023. [9] CISA et al. #StopRansomware: ALPHV BlackCat. Advisory AA23-353A, December 2023. [10] H. Pittman et al. fre-research: LLM-based adversary procedure OS translation pipeline. MITRE Corporation, January 2026. [11] A. Madaan et al. Self-Refine: Iterative Refinement with Self-Feedback. NeurIPS, 36, 2023. [12] T. Szymkiewicz. On the Numerical Analysis of the Floristic Composition of Plant Associations. Bull. Acad. Pol. Sci., 1934.
14
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
A Enriched Procedure-Graph Schema Each step node carries the thirteen attributes consumed by the matching layers. The schema below is the structure of the enriched control graph (29 nodes, 28 sequential edges, automated 9/12). The variant graph uses the identical schema with the Linux rule family and source_os=linux. { " gbse_schema_version ": "2.0- winlin", "sigma_library ": "sigma_rules_gbse .yml -- Windows W01 -W30 (30 rules)", "source ": "Windows G_c reconstructed from conversion_notes + de -wine + bitsadmin map", "metadata ": { "procedure_id ": " alphv_blackcat_windows_ctrl ", "source_os ": "windows", "graph_topology ": { "V": 29, "E_sequential ": 28 }, "enrichment ": "Algorithm 1: telemetry_expected -> telemetry_classes ", " automated_evaluation ": { "pass ": 9, "total ": 12, "auto ": 0.75 } }, "procedure ": { "action_sequence ": [ { "step_id ": 3, "technique_id ": "T1021 .001" , " parent_technique_id ": "T1021", "tactic ": [" initial_access "], "telemetry_classes ": [" file"," identity "," network "," process "], "execution_level ": "non -elevated", "privilege_context ": "user", "privilege_delta ": "drop", "sigma_rules ": [ { "rule_id ":"..." , "name ":" W03", "logsource ":" windows/process_creation", "logsource_category ":" process_creation", "attack_tag ":" T1021 .001" , "match_type ":" technique", "tele_overlap ":0.75 } ], "os_mappable ": true , "data_flow ": null , "omission_reason ": null , " win_command_provenance ": "conversion_note" } ] } }
B Reference Implementation (core) The functions below are the computational core: the overlap coefficient, the four-class telemetry parser (Algorithm 1), the layer predicates, the bipartite GED (Algorithm 2), and Sigma injection (Algorithm 3). The sentinel initialization on the cost matrix is the correctness-critical line. 1 2 3
import numpy as np from scipy.optimize import linear_sum_assignment INF = 1e9
4 5 6 7 8
def szss(A, B): # Szymkiewicz-Simpson overlap coefficient if not A and not B: return 1.0 d = max(len(set(A)), len(set(B))) return len(set(A) & set(B)) / d if d else 0.0
9 10
def lsc(ls): return ls.split("/")[-1] if "/" in ls else ls # logsource category
11 12 13 14 15 16 17 18 19
# Algorithm 1: free-text telemetry_expected →structured observable classes _KW = {"process": {"process","execut","spawn","binary","creation event"}, "network": {"network","tcp","http","ssh","scp","ldap","webdav","socket"}, "file": {"file","directory","disk","write","config","dump file"}, "identity":{"auth","credential","password","account","sudo","token"}} def parse_telemetry_expected(text): t = (text or "").lower() return sorted(c for c, kws in _KW.items() if any(k in t for k in kws))
20 21 22 23
# Layer predicates (each stacks on the previous) def m0(c, v): return c["technique_id"] == v["technique_id"] def m1(c, v): 15
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
if not m0(c, v): return False T1, T2 = set(c.get("tactic", [])), set(v.get("tactic", [])) return szss(T1, T2) ≥0.50 if (T1 and T2) else True def m2(c, v): # STRICT > 0.50 if not m1(c, v): return False C1, C2 = set(c["telemetry_classes"]), set(v["telemetry_classes"]) return szss(C1, C2) > 0.50 if (C1 or C2) else True def m3_chained(c, v): # requires L2 first if not m2(c, v): return False L1 = {lsc(r["logsource"]) for r in c.get("sigma_rules", [])} L2 = {lsc(r["logsource"]) for r in v.get("sigma_rules", [])} return szss(L1, L2) ≥0.50 if (L1 or L2) else True def m3_independent(c, v): # L0 + logsource only if not m0(c, v): return False L1 = {lsc(r["logsource"]) for r in c.get("sigma_rules", [])} L2 = {lsc(r["logsource"]) for r in v.get("sigma_rules", [])} return szss(L1, L2) ≥0.50 if (L1 or L2) else True
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55
# Algorithm 2: corrected bipartite GED (Riesen & Bunke, Hungarian assignment) def bipartite_ged(c_steps, v_steps, match_fn): nc, nv = len(c_steps), len(v_steps) C = np.full((nc + nv, nc + nv), INF) # <- sentinel init (the fix) for i, cs in enumerate(c_steps): for j, vs in enumerate(v_steps): C[i][j] = 0.0 if match_fn(cs, vs) else 1.0 # substitution for i in range(nc): C[i][nv + i] = 1.0 # deletion (diagonal) for j in range(nv): C[nc + j][j] = 1.0 # insertion (diagonal) for i in range(nv): for j in range(nc): C[nc + i][nv + j] = 0.0 # null-to-null row, col = linear_sum_assignment(C) ged = float(C[row, col].sum()) return ged, round(max(0.0, 1.0 - ged / max(nc, nv)), 4)
56 57 58 59 60 61 62 63 64 65
# Algorithm 3: attach a rule on technique-tag match OR telemetry overlap def inject_sigma(step, library): tech, tel = step["technique_id"], set(step.get("telemetry_classes", [])) out = [] for r in library: if tech in r["tags"] or szss(tel, set(r["tel"])) ≥0.50: out.append({"logsource": r["logsource"], "logsource_category": lsc(r["logsource"])}) return out
66 67 68 69 70
# Composite scoring def composite(auto, sim, tr, dv): bcf = round(0.5 * auto + 0.5 * sim, 4) return bcf, round(0.4 * bcf + 0.3 * tr + 0.3 * dv, 4)
C Sigma Rule Library Catalogue The complete 49-rule library is delivered in sigma_rules_gbse.yml. Table 7 lists every rule with its OS family, ATT&CK mapping, and severity. All rules use the process_creation logsource category. Table 7. The validated 49-rule Sigma library (19 Linux, 30 Windows). ID
OS
ATT&CK
Title
Level
L01
Linux
T1048.003
ALPHV rclone WebDAV Listener on Attacker Inf . . .
HIGH
continued on next page
16
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
ID
OS
ATT&CK
Title
Level
L02
Linux
T1105
CRITICAL
L03
Linux
T1021.001
L04
Linux
T1087.002
L05
Linux
T1018
L06
Linux
T1087.001
L07
Linux
T1105
L08
Linux
T1003.001,T1204.002
L09 L10
Linux Linux
T1562.001 T1112
L11
Linux
T1003.001
L12
Linux
T1048.003,T1071.001
L13 L14
Linux Linux
T1046,T1595.001 T1077
L15
Linux
T1059.001,T1204.002
L16 L17
Linux Linux
T1021.004,T1105 T1059.004
L18
Linux
T1083
L19
Linux
T1048.001
W01
Win
T1048.003
W02
Win
T1105
W03 W04
Win Win
T1021.001 T1087.002
W05 W06 W07 W08
Win Win Win Win
T1018 T1087.001 T1105,T1197 T1003.001
W09
Win
T1562.001
W10 W11
Win Win
T1112 T1003.001
W12 W13
Win Win
T1048.003 T1046,T1595.001
W14
Win
T1077
Attacker C2 Control Server Binary Execution ... SSH Interactive Bash Login Shell Used for La . . . ldapsearch Domain User Account Enumeration v... ldapsearch Domain Computer Account Discovery Local Account Enumeration via /etc/passwd Re . . . wget Executable Binary Download to Staging P... Wine Windows PE Execution for Credential Acc . . . systemctl Security Service Stop and Disable sysctl Kernel Security Parameter Modificatio . . . gcore Process Memory Dump for Credential Ext . . . rclone Data Exfiltration to Remote WebDAV En . . . nmap Ping Sweep Internal Network Discovery scp Batch Deployment to Multiple Internal Ho . . . Windows PE or Wine Binary Execution from Sta . . . scp Lateral File Transfer of Tool or Payload SSH Remote Shell with chmod and Ransomware E . . . smbclient Admin Share File Collection via Ge . . . scp Exfiltration to External AttackerContro . . . rclone WebDAV Server Started on Windows Host C2 Framework or RAT Binary Executed from Non . . . mstsc.exe RDP Connection to Internal Host PowerShell or nltest Domain User Account Enu . . . PowerShell AD Computer Discovery net.exe Local User Account Discovery bitsadmin PE Binary Staging Download InfoStealer LSASS Credential Access via Name . . . Security Software Process Terminate or Servi . . . Registry ASLR Exploit Mitigation Disable ProcDump or Task Manager LSASS Full Memory D . . . rclone Copy Exfiltration to Named Remote nmap or Network Port Scanner Execution on Wi . . . PsExec or Admin Share Lateral Tool Deploymen . . .
MEDIUM MEDIUM LOW LOW HIGH CRITICAL HIGH HIGH HIGH HIGH MEDIUM HIGH HIGH MEDIUM CRITICAL HIGH HIGH HIGH MEDIUM MEDIUM MEDIUM LOW LOW HIGH CRITICAL HIGH HIGH CRITICAL MEDIUM MEDIUM HIGH
continued on next page
17
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
ID
OS
ATT&CK
Title
Level
W15 W16 W17
Win Win Win
T1059.001,T1486 T1021.004,T1105 T1059.004,T1059.003
HIGH MEDIUM MEDIUM
W18
Win
T1083
W19 W20
Win Win
T1048.001 T1021.004,T1105,T1048.001
W21
Win
T1021.004,T1059.004
W22
Win
T1105
W23
Win
T1562.001,T1562.004
W24
Win
T1046
W25
Win
T1059.001,T1486
W26
Win
T1003.001,T1555
W27
Win
T1105,T1071.001
W28
Win
T1083,T1135
W29
Win
T1133
W30
Win
T1555,T1555.003
PowerShell or PE Execution from Temp Path pscp.exe or WinSCP SCP File Transfer CMD Inline Command via SSH Channel Remote Ex . . . net.exe Admin Share File Discovery and Colle . . . robocopy or xcopy External Host Exfiltration OpenSSH scp.exe Secure Copy Transfer or Exfi . . . OpenSSH ssh.exe Interactive Remote Command E . . . wget.exe or curl.exe LOLbin Ingress Tool Tra . . . Native Firewall Disable or Microsoft Defende . . . PowerSploit Invoke-Portscan Network Service ... PE Execution from User-Writable Temp Path or . . . Non-Standard InfoStealer Binary LSASS or Cre . . . Cross-Platform C2 Server Binary Execution fr . . . PowerShell Get-ChildItem Admin-Share File Di . . . External Remote Services Enablement or Expos . . . Credentials from Password Stores Access
MEDIUM HIGH HIGH HIGH MEDIUM HIGH MEDIUM HIGH HIGH MEDIUM MEDIUM HIGH HIGH
A representative rule, L01, shown in full to fix the format used throughout the library: title: ALPHV rclone WebDAV Listener on Attacker Infrastructure id: aa255d9f-995d-41a9-84ba-e4949f564c97 name: L01 status: experimental description: > Detects rclone started in WebDAV server mode, binding to a non-loopback address. Procedure S1: rclone serve webdav /srv/http --addr 176.59.1.18:8080 (T1048.003). Attacker hosts a WebDAV endpoint to receive exfiltrated data. author: Elmisery, Fujitsu Research of Europe date: 2026/06/05 tags: - attack.exfiltration - attack.t1048.003 - attack.command_and_control logsource: product: linux category: process_creation detection: selection_serve: Image|endswith: '/rclone' CommandLine|contains|all: ['serve', 'webdav'] filter_loopback: CommandLine|contains: ['--addr 127.', '--addr ::1', '--addr localhost'] condition: selection_serve and not filter_loopback falsepositives: - Legitimate rclone WebDAV mounts for cloud storage workflows level: high 18
Fujitsu Research of Europe · MITRE Research
Graph-Based Structural Evaluation
x-gbse-telemetry: [file, network, process] x-gbse-steps: [1]
Fujitsu Research of Europe Limited, Security Science Research Group. In collaboration with MITRE Research. This document describes a defensive evaluation methodology and the detection content it produced; all attack tradecraft referenced is drawn from public threat-intelligence reporting on ALPHV/BlackCat.
19