arXiv:2604.10800v1 [cs.SE] 12 Apr 2026
Verify Before You Fix: Agentic Execution Grounding for Trustworthy Cross-Language Code Analysis
Jugal Gajjar∗ Department of Computer Science The George Washington University Washington, DC 20052 [email protected]
Abstract Learned classifiers deployed in agentic pipelines face a fundamental reliability problem: predictions are probabilistic inferences, not verified conclusions, and acting on them without grounding in observable evidence leads to compounding failures across downstream stages. Software vulnerability analysis makes this cost concrete and measurable. We address this through a unified cross-language vulnerability lifecycle framework built around three LLM-driven reasoning stages— hybrid structural-semantic detection, execution-grounded agentic validation, and validation-aware iterative repair—governed by a strict invariant: no repair action is taken without execution-based confirmation of exploitability. Cross-language generalization is achieved via a Universal Abstract Syntax Tree (uAST) normalizing Java, Python, and C++ into a shared structural schema, combined with a hybrid fusion of GraphSAGE and Qwen2.5-Coder-1.5B embeddings through learned two-way gating, whose per-sample weights provide intrinsic explainability at no additional cost. The framework achieves 89.84–92.02% intra-language detection accuracy and 74.43–80.12% zero-shot cross-language F1, resolving 69.74% of vulnerabilities end-to-end at a 12.27% total failure rate. Ablations establish necessity: removing uAST degrades cross-language F1 by 23.42%, while disabling validation increases unnecessary repairs by 131.7%. These results demonstrate that execution-grounded closed-loop reasoning is a principled and practically deployable mechanism for trustworthy LLM-driven agentic AI.
1
Introduction
Large language models and learned classifiers have demonstrated remarkable capability across code understanding, medical diagnosis, legal reasoning, and financial decision-making, yet their deployment in high-stakes settings exposes a fundamental limitation: predictions are probabilistic inferences over training distributions, not verified conclusions about the world. Acting on such predictions without grounding them in observable evidence leads to compounding failures: false positives accumulate, downstream agents expend effort on non-issues, and human operators lose trust through repeated false alarms [39, 11]. How AI systems should handle uncertainty in consequential multi-stage decisions and communicate the basis for those decisions to human collaborators remains a central open problem in trustworthy and explainable AI. Software vulnerability analysis offers a uniquely rigorous testbed for these questions. Unlike many domains where ground truth is subjective or delayed, exploitability is verifiable through program execution—providing a concrete feedback signal that AI reasoning systems can use to ground their predictions. The task is inherently multi-stage: a detection model identifies candidates, a reasoning ∗ Alternate Email: [email protected]
Preprint.
agent confirms genuine exploitability, and a generative model produces a correct fix. Each stage demands a different reasoning modality, structural pattern recognition, hypothesis-driven execution reasoning, and constrained code generation, making it an ideal setting for studying how LLM-driven agents can reason reliably across a decision lifecycle [61, 35]. Yet existing approaches fragment this lifecycle into isolated tools: detectors predict without confirming, repair systems act on unverified findings [62], and all major approaches require separate models per programming language [25, 54]. We present a unified cross-language vulnerability lifecycle framework built around three LLM-driven reasoning stages: hybrid structural-semantic detection, execution-grounded agentic validation, and validation-aware iterative repair. The governing principle is that no repair action is taken without execution-based confirmation—converting probabilistic predictions into evidence-backed decisions. We argue this constitutes a general principle for trustworthy agentic AI: execution grounding as a mechanism for uncertainty reduction in multi-stage LLM pipelines. Cross-language generalization is achieved through a Universal Abstract Syntax Tree (uAST) [18] normalizing Java, Python, and C++ into a shared schema, combined with a hybrid fusion of graph embeddings [24] and Qwen2.5-Coder1.5B [28] semantic representations via learned two-way gating [17]. The per-sample gating weights are directly interpretable, revealing whether structural or semantic reasoning drove each prediction, providing intrinsic explainability at no additional inference cost. Our contributions are fourfold: 1. Universal structural representation for cross-language LLM reasoning. A uAST schema enabling zero-shot transfer across Java, Python, and C++ at 74.43–80.1% F1, a 23.42% improvement over language-specific representations without per-language retraining. 2. Interpretable hybrid reasoning with modality-aware gating. A two-way gating fusion of structural and LLM semantic reasoning, achieving 89.84–92.02% detection accuracy with per-sample explainability at no additional cost. 3. Execution-grounded agentic validation as a trustworthy AI mechanism. An LLM-driven agent that converts classifier predictions into execution-verified decisions. Ablations show removing this stage increases unnecessary repairs by 131.7% and reduces end-to-end success by 9.56 percentage points. 4. Closed-loop iterative repair with human-AI collaboration support. A validation-aware LLM repair loop acting exclusively on confirmed findings, with structured diagnostic traces surfaced to human reviewers for non-convergent cases. Together, these contributions demonstrate that execution-grounded feedback loops are a principled and empirically effective mechanism for reducing overconfidence in LLM-driven agentic pipelines— establishing a blueprint for trustworthy, explainable, and human-collaborative AI reasoning in high-stakes domains. All code, dataset splits, and precomputed embeddings will be released upon acceptance.
2
Related Work
2.1
Uncertainty and Evidence Grounding in Agentic AI
Probabilistic predictions in multi-stage pipelines accumulate errors when downstream agents act without external verification—a failure mode documented across LLM planning [56], self-refinement [27], and tool-augmented reasoning [63, 49, 21]. Verification-before-action has consequently emerged as a core principle in reliable agentic design, with grounded feedback from execution, proof-checkers, or tool outputs measurably improving decision quality over model-internal reasoning alone [21]. Software vulnerability analysis instantiates this problem concretely: exploitability is directly verifiable through execution [55], making it a uniquely rigorous testbed for evidence-grounded agentic AI. 2.2
LLMs for Code Understanding and Security Analysis
Early transformer-based code models—CodeBERT [15], GraphCodeBERT [22], CodeT5 [60]— established transfer learning for security tasks, while more recent autoregressive models including Code Llama [46], DeepSeek-Coder [23], and Qwen2.5-Coder [28] extend this with richer semantic reasoning over API usage and control-flow intent [65]. However, LLMs in security-critical contexts 2
Table 1: Comparison of representative approaches across five dimensions. ✓ full support, ◦ partial, × absent. Ours is the only approach achieving all five simultaneously; execution grounding is absent from all prior work. Multi-stage Lifecycle
Cross-lang. Transfer
Execution Grounding
Intrinsic XAI
Human–AI Collab.
Devign [66] GraphCodeBERT [22] VulGraB [58] LineVul [16] VulAgent [61] MAVUL [35] AutoPatch [48] SecureFixAgent [20] MalCodeAI [19]
× × × × ✓ ◦ ◦ ◦ ◦
× ◦ × × × × × × ✓
× × × × × × × × ×
× × × ◦ × × × × ×
× × × × × ◦ × × ×
Ours
✓
✓
✓
✓
✓
Approach
remain susceptible to hallucination [29, 64] and long-range context limitations [36], motivating hybrid architectures that complement semantic reasoning with structural program representations. 2.3
Reasoning-Centric and Agentic AI Systems
Multi-stage reasoning pipelines substantially outperform single-pass prediction in program analysis. VulAgent [61] and MAVUL [35] demonstrate that hypothesis generation and iterative refinement reduce false positives, while agentic CVE reproduction frameworks [55] show that plan-executeverify reasoning is essential for reliable conclusions. In repair, SAN2PATCH [31], AutoPatch [48], and SecureFixAgent [20] employ structured multi-step reasoning—yet all rely on static analysis for validation rather than execution-grounded evidence, leaving the exploitability confirmation gap unaddressed. 2.4
Structural Reasoning, Explainability, and Trustworthy AI
GNN-based detectors—Devign [66], ReVeal [8], VulGraB [58]—introduced structured program reasoning but remain language-specific [25, 54]. Hybrid structural-semantic fusion [17] shows complementary gains, yet prior explainability approaches rely on post-hoc attribution [16, 2] rather than intrinsic architectural transparency. Trustworthy deployment of learned classifiers requires moving beyond statistical predictions toward observable, grounded evidence [2], a gap that selfrefinement approaches [61, 35] address only partially, as corrections remain within the model’s own reasoning space rather than grounded in execution. 2.5
Automated Repair and Human-AI Collaboration
APR has evolved from template-based methods [34, 37] through neural models [10, 38] to LLMdriven patch generation [62, 3]. A persistent limitation across all paradigms is operating on unverified detector outputs [62]—repair effort is allocated without execution-based confirmation of exploitability, and non-convergent cases fail silently without supporting human oversight. 2.6
Our Positioning
Our framework enforces execution grounding as a governing invariant across all three lifecycle stages— inference, grounding, and action—unlike prior pipelines that act on unverified predictions [56, 27]. Unlike prior detection approaches, we achieve cross-language transfer through uAST normalization [18] without per-language retraining. Unlike prior agentic systems, our validation agent grounds predictions in observable execution behavior rather than model-internal reasoning. Unlike prior repair systems, remediation is gated on execution-confirmed exploitability and non-convergent cases are delegated to human reviewers with structured diagnostic traces, making human-AI collaboration a first-class design principle rather than an afterthought. 3
Figure 1: Three-stage lifecycle architecture. The Fusion Detector combines graph and LLM embeddings via two-way gating for binary detection. The Validation Agent confirms exploitability through sandboxed execution. The Remediation module generates minimal patches, outputting a detection flag, validation trace, and fixed code.
3
Proposed Framework
3.1
Overview
The framework instantiates a general principle for trustworthy agentic AI: a three-stage closed loop of inference, grounding, and action, where each stage gates the next on verified evidence rather than probabilistic prediction. This design is domain-agnostic—the lifecycle applies wherever predictions are verifiable and acting on unconfirmed outputs carries measurable cost. Figure 1 illustrates the three-stage lifecycle framework for our vulnerability analysis testbed. Given a source file, the detection stage parses it into a Universal Abstract Syntax Tree (uAST) and produces a binary vulnerability flag via hybrid fusion of graph and LLM embeddings. Flagged samples enter the validation stage, where language-specific agents—dispatched through a shared LLM planner, Docker sandbox, and evidence collector—confirm genuine exploitability through sandboxed execution. Confirmed samples proceed to iterative repair, where a fine-tuned LLM generates minimal patches under re-detection supervision, with structured diagnostic traces surfaced to human reviewers for non-convergent cases. The governing invariant is strict: no repair action is taken without executionbased confirmation. The following subsections detail the internals of each stage and how they operate within this lifecycle. 3.2
Universal AST and Hybrid Detection
Cross-language vulnerability detection faces a fundamental representation mismatch: identical vulnerability patterns manifest through radically different surface syntax across languages. We address this through a Universal Abstract Syntax Tree (uAST) that normalizes Java, Python, and C++ into a shared four-layer schema—metadata, flat node array, universal node taxonomy (47 categories over 200+ language-specific types), and cross-language semantic mapping—constructed via Tree-sitter [6] and serialized in Parquet format [18]. 4
The hybrid detection model fuses two complementary reasoning branches. The structural branch encodes each uAST as a directed graph processed by GraphSAGE [24], sampling fixed-size neighborhoods through mean aggregation across two message-passing layers to produce a 128-dimensional graph-level embedding. The semantic branch encodes raw source text through Qwen2.5-Coder-1.5B [28], averaging the final hidden states across sequence positions to produce a 1536-dimensional embedding. Both embeddings are L2-normalized and projected to a shared 128-dimensional latent space: ĝi = LayerNorm ϕ Wg h̃G,i + bg ,
ˆli = LayerNorm ϕ Wl h̃L,i + bl
(1)
A two-way gating mechanism computes per-sample modality scores via learned scoring networks, normalized through softmax to weights [αg,i , αl,i ] summing to one. The fused representation ĥi = αg,i ĝi + αl,i ˆli drives a binary classifier. These weights are directly interpretable: αg ≈ 1 for structurally explicit vulnerabilities such as buffer overflows, αl ≈ 1 for logic vulnerabilities requiring semantic context—providing per-prediction explainability intrinsic to the architecture. The model trains with composite loss: L = LCE + λnce LInfoNCE + λlap LLaplacian
(2)
where LInfoNCE [42] encourages modality alignment and LLaplacian [1] smooths structural embeddings over connected graph regions. 3.3
Execution-Grounded Agentic Validation
The validation agent (Figures 7–9) implements a plan-execute-verify reasoning loop that converts probabilistic detector outputs into binary, execution-backed verdicts. Given a flagged sample, an LLM planner (Qwen2.5-Coder-1.5B-Instruct) generates a structured exploit hypothesis specifying attack vector, concrete payloads, and environmental preconditions. A language-specific harness generator synthesizes instrumented test programs—integrating AddressSanitizer and UndefinedBehaviorSanitizer for C++, AspectJ [30] bytecode instrumentation for Java, and module-level monkeypatching for Python. Harnesses execute within Docker containers under strict resource isolation (memory limits, network isolation, system call monitoring, 30–45 second timeouts). The agent operates asymmetrically based on detection flags. For flag = 1, active confirmation deploys 3–5 payloads per hypothesis with a 60-second timeout. For flag = 0, limited probing applies 1–2 generic payloads with a 15-second timeout, recovering false negatives while minimizing overhead. Early stopping triggers immediately upon confirmed exploitation, with iterative refinement applied when initial tests are inconclusive—up to a maximum of five attempts. Evidence classification distinguishes confirming, suggestive, and neutral observations; at least one confirming item is required to declare exploitation success. 3.4
Validation-Aware Iterative Repair
Repair activates exclusively for execution-confirmed samples. A LoRA fine-tuned [26] Qwen2.5Coder-1.5B-Instruct generates minimal patches from structured prompts containing the vulnerable code, confirmed vulnerability type, successful exploit payload, and observed malicious behavior. Patch application uses AST manipulation libraries—redbaron [45] for Python, javaparser [52] for Java, libclang [44] for C++—ensuring syntactic validity and localized scope. Re-detection parses the patched code into uAST and re-evaluates through the full hybrid model; flag = 0 terminates the loop with success, flag = 1 triggers the next iteration with accumulated context from prior attempts. The loop runs for a maximum of five iterations. Successfully repaired samples are verified through differential analysis confirming that only vulnerability-implicated regions were modified. Nonconvergent samples, those exhausting the iteration budget, are flagged for human review with structured diagnostic traces including all attempted patches, rejection reasons, and persistent vulnerability indicators, positioning the system as a human-AI collaborative tool at the boundary of autonomous capability. 5
4
Experiments and Results
4.1
Experimental Setup
Experiments are conducted across Java, Python, and C++ on a balanced dataset of 120,000 code files, 60,000 vulnerable (paired with fixes) and 60,000 safe, distributed equally at 40,000 samples per language, split 80/10/10 for train/validation/test. The dataset integrates real-world vulnerability databases (PrimeVul [14], MegaVul [41], DiverseVul [9], CVEfixes [4], ReposVul [59], Vul4J [7], PySecDB [51], among others; see Appendix A) with ~20% synthetic samples generated by GPT4.1 and Claude Sonnet 4.5 to ensure balanced vulnerability type coverage across SQL injection, command injection, path traversal, and insecure deserialization. The selected detection architecture— GraphSAGE [24] paired with Qwen2.5-Coder-1.5B [28]—was chosen through systematic evaluation of nine combinations (3 graph encoders × 3 LLMs), achieving the highest hybrid F1 of 90.21% (Appendix D). All experiments run on Google Colab Pro with NVIDIA A100/H100 GPUs; local inference uses an Apple M4 MacBook Pro (48GB) via the MLX framework. 4.2
Detection Results
Table 2: Zero-shot cross-language detection results. The hybrid model outperforms both baselines across all six pairs, with C++ → Java achieving the highest F1 (0.7818). Structural-only transfer degrades toward near-random, confirming uAST normalization as essential for cross-language generalization. Structural-Only
Semantic-Only
Hybrid
Train → Test
F1
Acc
F1
Acc
F1
Acc
Java → Python Java → C++ Python → Java Python → C++ C++ → Java C++ → Python
0.5521 0.5287 0.5734 0.5469 0.5196 0.5348
59.34% 57.12% 61.03% 58.41% 56.48% 57.36%
0.6824 0.7016 0.6892 0.6984 0.7129 0.7041
71.86% 73.42% 72.51% 72.94% 74.08% 73.27%
0.7443 0.7692 0.7525 0.7716 0.7818 0.7593
76.84% 78.97% 77.63% 79.12% 80.12% 78.45%
Average
0.5426
58.29%
0.6981
73.01%
0.7631
78.52%
The hybrid model achieves 89.84–92.02% intra-language accuracy and 0.8837–0.9109 F1 across all three languages (Figure 2), outperforming structural-only and semantic-only baselines by 8–15 percentage points. Cross-language transfer (Table 2) demonstrates meaningful zero-shot generalization at 74.43–80.12% F1—a 23.42% average improvement over language-specific representations. C++ achieves the highest intra-language accuracy (92.02%), reflecting explicit structural patterns in memory management vulnerabilities. Semantic-only models exhibit poor cross-language transfer, confirming that uAST structural normalization is the primary driver of generalization. 4.3
Validation and Repair
Table 3: Validation effectiveness across languages. For flag = 1, the agent confirms exploitability in 66.84–71.49% of cases while rejecting 58.72–62.37% of false positives. For flag = 0, limited probing recovers 11.76–16.21% of missed vulnerabilities at a spurious rate of only 2.67–3.98%. Flag = 1
Flag = 0
Language
Exploit Confirm
FP Rejection
FN Recovery
Spurious Rate
Java Python C++
70.28% 71.49% 66.84%
60.43% 62.37% 58.72%
13.87% 16.21% 11.76%
2.94% 2.67% 3.98%
Average
69.54%
60.51%
13.95%
3.20%
Validation confirms 66.84–71.49% of flagged samples as genuinely exploitable while rejecting 58.72– 62.37% of detector false positives (Table 3), preventing unnecessary repair effort on non-exploitable 6
Figure 2: Intra-language detection performance. The hybrid model achieves 89.84–92.02% accuracy and 0.8837–0.9109 F1, outperforming both baselines by 8–15 percentage points. C++ achieves the highest accuracy (92.02%), reflecting explicit structural patterns in memory management vulnerabilities. Table 4: Repair performance across languages. Python achieves the highest success rate (87.27%, 2.3 iterations) while C++ is most challenging (81.37%, 3.4 iterations). Post-repair pass rates of 90.44–93.15% confirm patches robustly eliminate vulnerabilities. Language
Repair Success
Avg Iterations
Max-Iter Failures
Post-Repair Pass
Java Python C++
85.14% 87.27% 81.37%
2.7 2.3 3.4
10.38% 8.17% 14.79%
91.36% 93.15% 90.44%
Combined
83.78%
2.8
11.12%
91.63%
findings. Python achieves the highest confirmation rate through dynamic typing simplifying automated test construction; C++ exhibits the lowest, reflecting the complexity of triggering memory-level exploits. Repair succeeds in 81.37–87.27% of cases within the five-iteration budget (Table 4), converging in 2.3–3.4 iterations on average. Post-repair detection pass rates of 90.44–93.15% confirm patches robustly eliminate vulnerabilities. Language difficulty follows Python (easiest), Java (moderate), C++ (hardest)—consistent with manual repair complexity. 4.4
End-to-End Performance and Ablation
The full pipeline resolves 69.74% of vulnerabilities end-to-end at a 12.27% total failure rate (Figure 3), with validation eliminating 61.24% of detector false positives before remediation and avoiding 73.13% of unnecessary repairs. The 69.74% end-to-end resolution reflects compounding multi-stage error rates—each stage operating at high but imperfect accuracy—rather than any single point of failure. Ablation studies (Table 5) validate the necessity of both core components. Removing uAST normalization degrades average cross-language F1 by 23.42% while incurring only 3.79% intra-language loss, confirming it as the primary driver of cross-language generalization. Disabling execution-grounded validation increases unnecessary repairs by 131.7% and reduces end-to-end success by 9.56 percentage points, directly quantifying the cost of prediction-only pipelines in highstakes agentic settings. The full system processes 1,700–2,400 samples daily on a single A100 GPU (37–49 seconds per sample), enabling practical CI/CD integration without cloud dependencies.
5
Discussion
5.1
Execution Grounding as a General Trustworthy AI Principle
Our ablation results, +131.7% unnecessary repairs and −9.56 percentage points end-to-end success without validation, quantify a cost that extends beyond vulnerability analysis: agentic LLM pipelines acting on probabilistic predictions without external verification systematically accumulate errors 7
Table 5: End-to-end results and ablation study. Full pipeline resolves 69.74% of vulnerabilities at 12.27% failure rate. Removing uAST degrades cross-language F1 by 23.42%; disabling validation increases unnecessary repairs by 131.7% and reduces end-to-end success by 9.56 pp—establishing both components as essential. Configuration
Metric
Value
Full system (end-to-end pipeline) Resolved vulnerabilities False positives eliminated Unnecessary repairs avoided Total pipeline failure
69.74% 61.24% 73.13% 12.27%
Ablation A: remove uAST normalization Avg intra-language F1 drop Avg cross-language F1 drop
−3.79% −23.42%
Ablation B: disable execution-grounded validation Unnecessary repairs (avg) End-to-end success drop
+131.7% −9.56 pp
Figure 3: End-to-end pipeline metrics. The full system resolves 69.74% of vulnerabilities, eliminates 61.24% of false positives, avoids 73.13% of unnecessary repairs, and maintains a 12.27% total failure rate—demonstrating stable multi-stage integration. across stages. This mirrors failure modes documented in tool-augmented agents [61, 35] and selfrefinement systems, where corrections remain within the model’s own reasoning space rather than grounded in observable outcomes. Our plan-execute-verify loop demonstrates that introducing a single execution-grounded verification stage between prediction and action measurably improves reliability—a principle transferable to any agentic pipeline operating in domains with verifiable ground truth, including theorem proving, scientific hypothesis testing, and code generation with test-based feedback [21]. More broadly, any domain where a binary oracle exists—unit tests, formal checkers, simulators, or experimental outcomes—can instantiate this loop with domain-specific grounding mechanisms in place of execution. 5.2
Intrinsic Explainability through Architectural Fusion
The per-sample gating weights (αg , αl ) produced by our two-way fusion mechanism reveal which reasoning modality—structural or semantic—drove each detection decision, without any post-hoc attribution overhead. This contrasts with the dominant XAI paradigm of applying gradient-based or attention-based explanation methods after prediction [8, 16], which are known to be unstable under input perturbations [2]. Our results show that αg ≈ 1 for structurally explicit vulnerabilities and αl ≈ 1 for semantic logic flaws, a consistent, faithful signal that arises naturally from the architecture. This suggests that designing fusion mechanisms with interpretable weighting is a practical path toward intrinsic XAI in any multi-modal reasoning system—from medical imaging fused with clinical text 8
to code structure fused with semantics—more faithful than post-hoc methods applied to black-box models. 5.3
Structural Abstraction for Cross-Domain Transfer in Learned Systems
The 23.42% cross-language F1 improvement from uAST normalization, with only 3.79% intralanguage cost, illustrates a general representation learning principle: task-relevant structural abstraction enables transfer that purely semantic representations cannot achieve alone. Semantic-only models fail at cross-language transfer because they encode surface lexical patterns that do not generalize across syntactic conventions. The uAST addresses this by lifting language-specific constructs into a shared structural schema, analogous to how intermediate representations enable transfer in other domains, compiler IRs across hardware targets [33], universal dependency parsing across natural languages, and modality-agnostic embeddings in multimodal learning. Learned systems operating across heterogeneous input domains consistently benefit from such abstraction layers, and our results provide concrete empirical grounding for this principle in the code reasoning setting. 5.4
Limitations and Future Work
Three limitations bound the current framework—and reflect broader challenges in execution-grounded agentic systems generally: bounded reasoning horizon in multi-hop inference chains (2-hop GraphSAGE insufficient for complex taint flows), incomplete environmental reconstruction in sandboxed verification (producing false negatives on complex deserialization and format string vulnerabilities), and coverage gaps in evaluation (concurrency, cryptographic misuse, and business logic flaws remain unevaluated). Looking forward, training the validation agent via reinforcement learning with exploit confirmation as a reward signal could improve discovery rates for complex patterns. Expanded language coverage is immediately feasible as Tree-sitter provides grammars for 40+ languages and the uAST schema is already mapped for ten [18]. Integrating formal verification for patch correctness would elevate repair from vulnerability removal to provably safe patch synthesis, closing the remaining gap between autonomous AI capability and production deployment. Future research in execution-grounded agentic AI should prioritize addressing these limitations to extend such frameworks beyond security toward broader high-stakes domains.
6
Conclusion
We presented a unified cross-language vulnerability lifecycle framework demonstrating that executiongrounded feedback loops are a principled and empirically effective mechanism for reducing overconfidence in LLM-driven agentic pipelines. By integrating hybrid structural-semantic detection, autonomous execution-grounded validation, and validation-aware iterative repair into a closed-loop system, we achieve 89.84–92.02% intra-language detection accuracy, 74.43–80.12% zero-shot crosslanguage F1, and 69.74% end-to-end vulnerability resolution with ablations directly quantifying the cost of removing each component. Beyond software security, the core architectural principles established here—execution grounding as uncertainty reduction, structural abstraction for cross-domain transfer, and intrinsic explainability through fusion design—are broadly applicable to any agentic AI system operating in high-stakes domains where predictions must be verified before action is taken. As LLM-driven agents are increasingly deployed in consequential multi-stage decision pipelines, building in mechanisms that know when to verify, when to repair, and when to defer to human judgment becomes not an engineering detail but a fundamental requirement for trustworthy AI.
References [1] Rie Ando and Tong Zhang. Learning on graph with laplacian regularization. Advances in neural information processing systems, 19, 2006. [2] Daniel Arp, Erwin Quiring, and Feargus et al. Pendlebury. Dos and don’ts of machine learning in computer security. In 31st USENIX Security Symposium (USENIX Security 22), pages 3971–3988, 2022. [3] Guru Bhandari, Nikola Gavric, and Andrii Shalaginov. Generating vulnerability security fixes with code language models. Information and Software Technology, 185:107786, 2025. 9
[4] Guru Bhandari, Amara Naseer, and Leon Moonen. Cvefixes: automated collection of vulnerabilities and their fixes from open-source software. In Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering, pages 30–39, 2021. [5] Sitanath Biswas. Vulnerability-fix-dataset: A curated repository for training and evaluating automated vulnerability remediation models. Kaggle, https://www.kaggle.com/datasets/ jiscecseaiml/vulnerability-fix-dataset, 2026. [6] Max Brunsfeld, Amaan Qureshi, and Andrew Hlynskyi et al. tree-sitter/tree-sitter: v0.26.6. Zenodo, 2026. [7] Quang-Cuong Bui, Riccardo Scandariato, and Nicolás E Díaz Ferreyra. Vul4j: A dataset of reproducible java vulnerabilities geared towards the study of program repair techniques. In Proceedings of the 19th International Conference on Mining Software Repositories, pages 464–468, 2022. [8] Saikat Chakraborty, Rahul Krishna, and Yangruibo et al. Ding. Deep learning based vulnerability detection: Are we there yet? IEEE Transactions on Software Engineering, 48(9):3280–3296, 2021. [9] Yizheng Chen, Zhoujie Ding, and Lamya et al. Alowain. Diversevul: A new vulnerable source code dataset for deep learning based vulnerability detection. In Proceedings of the 26th international symposium on research in attacks, intrusions and defenses, pages 654–668, 2023. [10] Zimin Chen, Steve Kommrusch, and Michele et al. Tufano. Sequencer: Sequence-tosequence learning for end-to-end program repair. IEEE Transactions on Software Engineering, 47(9):1943–1959, 2019. [11] Margaret Chustecki. Benefits and risks of ai in health care: narrative review. Interactive Journal of Medical Research, 13(1):e53616, 2024. [12] Cmonplz. Python vulnerability remediation: A specialized dataset for instruction-tuning llms on python security patches. Hugging Face Datasets, https://huggingface.co/datasets/ cmonplz/Python_Vulnerability_Remediation, 2026. Code_vulnerability_security_dpo: A preference-aligned dataset for [13] CyberNative. securing code generation through direct preference optimization. Hugging Face Datasets, https://huggingface.co/datasets/CyberNative/Code_Vulnerability_ Security_DPO, 2026. [14] Yangruibo Ding, Yanjun Fu, and Omniyyah et al. Ibrahim. Vulnerability detection with code language models: How far are we? arXiv preprint arXiv:2403.18624, 2024. [15] Zhangyin Feng, Daya Guo, and Duyu et al. Tang. Codebert: A pre-trained model for programming and natural languages. In Findings of the association for computational linguistics: EMNLP 2020, pages 1536–1547, 2020. [16] Michael Fu and Chakkrit Tantithamthavorn. Linevul: A transformer-based line-level vulnerability prediction. In Proceedings of the 19th international conference on mining software repositories, pages 608–620, 2022. [17] Jugal Gajjar, Kaustik Ranaware, and Kamalasankari Subramaniakuppusamy. Bridging semantics & structure for software vulnerability detection using hybrid network models. arXiv preprint arXiv:2510.10321, 2025. [18] Jugal Gajjar and Kamalasankari Subramaniakuppusamy. Mlcpd: A unified multi-language code parsing dataset with universal ast schema. arXiv preprint arXiv:2510.16357, 2025. [19] Jugal Gajjar, Kamalasankari Subramaniakuppusamy, and Noha El Kachach. Malcodeai: Autonomous vulnerability detection and remediation via language agnostic code reasoning. In 2025 IEEE International Conference on Information Reuse and Integration and Data Science (IRI), pages 31–36. IEEE, 2025. 10
[20] Jugal Gajjar, Kamalasankari Subramaniakuppusamy, and Relsy et al. Puthal. Securefixagent: A hybrid llm agent for automated python static vulnerability repair. arXiv preprint arXiv:2509.16275, 2025. [21] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738, 2023. [22] Daya Guo, Shuo Ren, and Shuai et al. Lu. Graphcodebert: Pre-training code representations with data flow. arXiv preprint arXiv:2009.08366, 2020. [23] Daya Guo, Qihao Zhu, and Dejian et al. Yang. Deepseek-coder: when the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024. [24] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017. [25] Nima Shiri Harzevili, Alvine Boaye Belle, and Junjie et al. Wang. A survey on automated software vulnerability detection using machine learning and deep learning. arXiv preprint arXiv:2306.11673, 2023. [26] Edward J Hu, Yelong Shen, and Phillip et al. Wallis. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3, 2022. [27] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798, 2023. [28] Binyuan Hui, Jian Yang, and Zeyu et al. Cui. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024. [29] Ziwei Ji, Nayeon Lee, and Rita et al. Frieske. Survey of hallucination in natural language generation. ACM computing surveys, 55(12):1–38, 2023. [30] Gregor Kiczales, Erik Hilsdale, Jim Hugunin, Mik Kersten, Jeffrey Palm, and William G Griswold. An overview of aspectj. In European Conference on Object-Oriented Programming, pages 327–354. Springer, 2001. [31] Youngjoon Kim, Sunguk Shin, and Hyoungshick et al. Kim. Logs in, patches out: Automated vulnerability repair via tree-of-thought llm analysis. In 34th USENIX Security Symposium (USENIX Security 25), pages 4401–4419, 2025. [32] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016. [33] Chris Lattner and Vikram Adve. Llvm: A compilation framework for lifelong program analysis & transformation. In International symposium on code generation and optimization, 2004. CGO 2004., pages 75–86. IEEE, 2004. [34] Claire Le Goues, ThanhVu Nguyen, and Stephanie et al. Forrest. Genprog: A generic method for automatic software repair. Ieee transactions on software engineering, 38(1):54–72, 2011. [35] Youpeng Li, Kartik Joshi, and Xinda et al. Wang. Mavul: Multi-agent vulnerability detection via contextual reasoning and interactive refinement. arXiv preprint arXiv:2510.00317, 2025. [36] Nelson F Liu, Kevin Lin, and John et al. Hewitt. Lost in the middle: How language models use long contexts. Transactions of the association for computational linguistics, 12:157–173, 2024. [37] Fan Long and Martin Rinard. Automatic patch generation by learning correct code. In Proceedings of the 43rd annual ACM SIGPLAN-SIGACT symposium on principles of programming languages, pages 298–312, 2016. 11
[38] Thibaud Lutellier, Hung Viet Pham, and Lawrence et al. Pang. Coconut: combining contextaware neural translation models using ensemble for program repair. In Proceedings of the 29th ACM SIGSOFT international symposium on software testing and analysis, pages 101–114, 2020. [39] Diego Marcilio, Rodrigo Bonifácio, and Eduardo et al. Monteiro. Are static analysis violations really fixed? a closer look at realistic usage of sonarqube. In 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC), pages 209–219. IEEE, 2019. [40] MarioVar. vulnerable-code_chitchat_doss1232: A conversational dataset for instructiontuning security-focused dialogue agents. Hugging Face Datasets, https://huggingface.co/ datasets/MarioVar/vulnerable-code_chitchat_doss1232, 2026. [41] Chao Ni, Liyu Shen, and Xiaohu et al. Yang. Megavul: Ac/c++ vulnerability dataset with comprehensive code representations. In Proceedings of the 21st International Conference on Mining Software Repositories, pages 738–742, 2024. [42] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. [43] Abdellah Oumida and Mohammed Sbaihi. Code_vulnerability_labeled_dataset. Hugging Face Datasets, https://huggingface.co/datasets/lemon42-ai/Code_Vulnerability_ Labeled_Dataset, 2025. [44] LLVM Project. Clang: A c language family frontend for llvm. https://clang.llvm.org/, 2026. [45] RedBaron Project. Redbaron: A bottom-up approach to fst (full syntax tree) for python. https://github.com/PyCQA/redbaron, 2026. [46] Baptiste Roziere, Jonas Gehring, and Fabian et al. Gloeckle. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023. [47] Marat Saratov. Source code vulnerability: A compact collection of labeled vulnerable code in c++, java, python, and more. Kaggle, https://www.kaggle.com/datasets/ maratsaratov/source-code-vulnerability, 2026. [48] Minjae Seo, Wonwoo Choi, and Myoungsung et al. You. Autopatch: Multi-agent framework for patching real-world cve vulnerabilities. arXiv preprint arXiv:2505.04195, 2025. [49] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652, 2023. [50] Shyyshawarma. cppvul: A curated dataset of c++ vulnerabilities for security-oriented language model training. Hugging Face Datasets, https://huggingface.co/datasets/ Shyyshawarma/cppvul, 2026. [51] Shiyu Sun, Shu Wang, and Xinda et al. Wang. Exploring security commits in python. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 171–181. IEEE, 2023. [52] JavaParser Team. Javaparser: A set of libraries for analyzing, transforming, and generating java source code. https://github.com/javaparser/javaparser, 2026. [53] Sunny Thakur. Vulnerable programming dataset: A comprehensive collection of 550 unique code vulnerabilities across 10 programming languages. Kaggle, https://www.kaggle.com/ datasets/cyberprince/vulnerable-programming-dataset, 2026. [54] Md Nizam Uddin, Yihe Zhang, and Xiali Hei. Deep learning aided software vulnerability detection: A survey. arXiv preprint arXiv:2503.04002, 2025. [55] Saad Ullah, Praneeth Balasubramanian, and Wenbo et al. Guo. From cve entries to verifiable exploits: An automated multi-agent framework for reproducing cves. arXiv preprint arXiv:2509.01835, 2025. 12
[56] Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Advances in Neural Information Processing Systems, 36:38975–38987, 2023. [57] Petar Veličković, Guillem Cucurull, and Arantxa et al. Casanova. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017. [58] Sixuan Wang, Chen Huang, and Dongjin et al. Yu. Vulgrab: Graph-embedding-based code vulnerability detection with bi-directional gated graph neural network. Software: Practice and Experience, 53(8):1631–1658, 2023. [59] Xinchen Wang, Ruida Hu, and Cuiyun et al. Gao. Reposvul: A repository-level high-quality vulnerability dataset. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, pages 472–483, 2024. [60] Yue Wang, Weishi Wang, and Shafiq et al. Joty. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 8696–8708, 2021. [61] Ziliang Wang, Ge Li, and Jia et al. Li. Vulagent: Hypothesis-validation based multi-agent vulnerability detection. arXiv preprint arXiv:2509.11523, 2025. [62] Boyang Yang, Zijian Cai, and Fengling et al. Liu. A survey of llm-based automated program repair: Taxonomies, design paradigms, and applications. arXiv preprint arXiv:2506.23749, 2025. [63] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations, 2022. [64] Ziyao Zhang, Chong Wang, and Yanlin et al. Wang. Llm hallucinations in practical code generation: Phenomena, mechanism, and mitigation. Proceedings of the ACM on Software Engineering, 2(ISSTA):481–503, 2025. [65] Xin Zhou, Sicong Cao, and Xiaobing et al. Sun. Large language model for vulnerability detection and repair: Literature review and the road ahead. ACM Transactions on Software Engineering and Methodology, 34(5):1–31, 2025. [66] Yaqin Zhou, Shangqing Liu, and Jingkai et al. Siow. Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Advances in neural information processing systems, 32, 2019.
13
A
Dataset Details
Table 6 provides the complete dataset composition. The full dataset integrates 15 real-world vulnerability databases and repositories supplemented with synthetic samples. Approximately 80% of samples (96,000) are sourced from real-world repositories and vulnerability databases; the remaining 20% (24,000) are synthetically generated using GPT-4.1 and Claude Sonnet 4.5 to ensure balanced representation across vulnerability types and coding patterns. All samples are distributed equally across Java, Python, and C++ at 40,000 samples per language (20,000 vulnerable + 20,000 safe). Table 6: Complete dataset sources by language coverage. ✓ denotes inclusion. The dataset integrates 15 real-world sources across curated vulnerability databases, repository-mined commits, and synthetic generation, ensuring comprehensive coverage of SQL injection, command injection, path traversal, and insecure deserialization across all three languages.
B
Source
Java
Python
C++
PrimeVul [14] MegaVul [41] DiverseVul [9] Vul4J [7] PySecDB [51] ReposVul [59] CVEfixes [4] Source Code Vulnerability [47] Vulnerable Programming Dataset [53] Vulnerability Fix Dataset [5] cmonplz/Python_Vulnerability_Remediation [12] Shyyshawarma/cppvul [50] MarioVar/vulnerable-code_chitchat_doss1232 [40] CyberNative/Code_Vulnerability_Security_DPO [13] lemon42-ai/Code_Vulnerability_Labeled_Dataset [43]
✓ ✓
✓ ✓
✓ ✓
✓
GPT-4.1 synthetic Claude Sonnet 4.5 synthetic
✓ ✓
✓ ✓
✓ ✓
✓ ✓ ✓ ✓
✓ ✓ ✓ ✓ ✓
✓ ✓ ✓ ✓
✓ ✓ ✓ ✓
Universal AST Schema
The uAST organizes each source file into four hierarchical layers designed around core properties like losslessness (preserving every syntactic element without semantic compression), uniformity (enforcing a consistent JSON schema across all languages), and queryability (enabling direct node addressing and category-based indexing without recursive traversal). The metadata layer captures global file characteristics, including language, node count, and a SHA-256 content hash for deduplication. The flat node array linearizes the tree into O(1)addressable nodes, each storing type, text content, source span, parent index, and child indices. The universal node taxonomy abstracts 200+ language-specific types into 47 universal categories—FUNCTION_DECLARATION, VARIABLE_ASSIGNMENT, CONTROL_FLOW, and others— enabling Python’s def, Java’s public static void, and C++’s function definitions to be queried uniformly. The cross-language semantic mapping layer translates language-specific constructs (Python’s with, Java’s try-with-resources, C++’s RAII) into shared universal roles while preserving original syntax. Construction uses Tree-sitter [6], selected over rule-based and regex-based approaches for its complete syntactic coverage, uniform cross-language interface, and robust error recovery. The pipeline comprises six deterministic stages: language detection, recursive AST extraction, node categorization, cross-language mapping, schema validation, and Parquet serialization. Processing throughput averages 600–800 files per minute on multi-core systems. 14
C
Implementation Details
The framework is implemented in Python 3.11, totaling approximately 13,000 lines organized into modular components. Core dependencies include PyTorch 2.9.0, Tree-sitter 0.21.0 [6], Transformers 4.57.1, and Docker 29.2.0. The GraphSAGE [24] encoder comprises two SAGEConv layers (PyTorch Geometric) with mean aggregation, sampling up to ten neighbors per node. The first layer maps 768-dimensional input features to 256-dimensional hidden representations; the second produces 128dimensional node embeddings. Global mean pooling aggregates node embeddings into graph-level representations with Xavier uniform initialization and batch normalization after each aggregation step. Semantic embeddings are extracted from Qwen2.5-Coder-1.5B in fp16 precision with 4,096 token maximum sequence length. For files exceeding this limit, a sliding window approach processes overlapping segments and averages embeddings. The fusion model trains with AdamW (learning rate 10−3 , weight decay 10−4 , batch size 64), with early stopping on validation F1 (patience 5 epochs, typically triggering around epoch 25–30). Contrastive loss temperature τ = 0.07; regularization weights λnce = 0.1, λlap = 0.01. Docker sandbox configurations enforce strict resource isolation: 1–2GB memory limits (languagedependent), 0.9 CPU core quota, –network=none isolation, read-only project mounts, –tmpfs /tmp:size=256m, PID limit of 256, and no-new-privileges security option. Standard streams are captured and truncated to 240KB. The LoRA fine-tuned repair model uses rank 16, adding only 12MB trainable parameters over the base Qwen2.5-Coder-1.5B-Instruct, fine-tuned on 60,000 vulnerability-patch pairs with temperature 0.2 and top-p 0.9 at inference.
D
Architecture Selection
Table 7: Hybrid model architecture selection across nine combinations of graph encoder and language model backbone. F1 scores reported on the combined validation set. GraphSAGE paired with Qwen2.5-Coder-1.5B achieves the highest F1 (90.21%) and is selected for all experiments. GraphSAGE consistently outperforms GCN and GAT across all LLM configurations through inductive neighborhood aggregation suited to irregular node-degree distributions in uAST graphs.
E
Graph Encoder
Qwen3-0.6B
DeepSeek-Coder-1.3B
Qwen2.5-Coder-1.5B
GCN [32] GraphSAGE [24] GAT [57]
0.8524 0.8915 0.8673
0.8607 0.8858 0.8843
0.8749 0.9021 0.8926
Detailed Experimental Results
Table 8: Intra-language detection accuracy and F1 scores. The hybrid model consistently achieves the highest accuracy across all languages, confirming complementary contributions of structural and semantic modalities. C++ achieves the highest hybrid accuracy (92.02%), reflecting explicit structural patterns in memory management vulnerabilities. Structural-Only
Semantic-Only
Language
Acc
F1
Acc
F1
Acc
F1
Java Python C++
81.56% 79.74% 84.28%
0.7992 0.7808 0.8211
77.90% 80.13% 72.87%
0.7584 0.7897 0.7043
90.23% 89.84% 92.02%
0.8901 0.8837 0.9109
15
Hybrid
Figure 4: Architecture selection grid comparing graph encoders and LLM backbones across nine combinations. GraphSAGE + Qwen2.5-Coder-1.5B achieves the highest F1 (90.21%, marked with ⋆) and is selected for all experiments. Table 9: Per-language repair performance. Python achieves highest success (87.27%) and fastest convergence (2.3 iterations); C++ is most challenging (81.37%, 3.4 iterations). Post-repair detection pass rates of 90.44–93.15% confirm patches robustly eliminate vulnerabilities.
F
Language
Repair Success
Avg Iterations
Max-Iter Failures
Post-Repair Pass
Java Python C++
85.14% 87.27% 81.37%
2.7 2.3 3.4
10.38% 8.17% 14.79%
91.36% 93.15% 90.44%
Combined
83.78%
2.8
11.12%
91.63%
Language-Specific Validation Agent Architectures
Figures 7, 8, and 9 provide the complete workflow diagrams for each language-specific validation agent. While all three share the same plan-execute-verify loop and evidence classification logic, their instrumentation strategies differ significantly across language runtimes. The Python agent leverages Python’s dynamic nature for comprehensive runtime instrumentation. Database monitoring patches sqlite3.Cursor.execute at module level; file access tracking instruments the built-in open function through monkeypatching; command injection detection wraps os.system and subprocess functions. Exception analysis captures full stack traces with local variable contexts on uncaught exceptions. The Java agent employs JVM-level instrumentation. JDBC query logging instruments PreparedStatement and Statement classes through AspectJ [30] bytecode weaving. Reflection monitoring tracks Method.invoke, Constructor.newInstance, and Field.set operations. A restrictive SecurityManager detects unauthorized filesystem operations, network connections, and native library loading. Heap analysis traces tainted data propagation through object references via the JVM Tool Interface. The C++ agent integrates compiler-level sanitizers. AddressSanitizer (compiled with -fsanitize=address) detects heap/stack/global buffer overflows, use-after-free, and double-free. UndefinedBehaviorSanitizer catches signed integer overflow, null pointer dereference, and misaligned memory access. Library interposition via LD_PRELOAD replaces security-sensitive functions (strcpy, strcat, system, popen, fopen, mysql_query) logging all invocations before delegating to original implementations.
16
Table 10: Average runtime per sample across pipeline stages. Validation dominates total execution time despite early stopping optimizations, reflecting Docker container startup, compilation, and harness execution overhead. Full pipeline runtime of 37.4–49.3 seconds per sample enables processing approximately 1,700–2,400 samples per day on a single A100 GPU. Language
Detection (s)
Validation (s)
Repair (s)
Total (s)
Java Python C++
0.18 0.16 0.21
28.6 24.8 34.0
13.6 12.4 15.1
42.38 37.36 49.31
Figure 5: Validation performance across languages. (a) Flag = 1: exploit confirmation rates of 66.84– 71.49%, false positive rejection of 58.72–62.37%, and early stopping frequencies of 65.13–76.44%. (b) Flag = 0: missed vulnerability recovery of 11.76–16.21% at spurious exploit rates of 2.67–3.98%. (c) Average tests before stopping: 2.5–3.4 across languages.
Figure 6: Repair performance metrics across languages. (a) Success rates: 81.37–87.27%. (b) Average iterations to convergence: 2.3–3.4. (c) Max-iteration failures: 8.17–14.79%. (d) Post-repair detection pass rates: 90.44–93.15%. 17
Figure 7: Python validation agent workflow showing hypothesis generation, payload construction, sandboxed execution with runtime monkeypatching instrumentation, and exploit confirmation through behavior observation.
18
Figure 8: Java validation agent workflow utilizing bytecode instrumentation via AspectJ [30], JDBC query logging, Security Manager integration, and heap analysis for tainted data propagation tracking.
19
Figure 9: C++ validation agent workflow utilizing AddressSanitizer, UndefinedBehaviorSanitizer, and library call interposition via LD_PRELOAD for memory safety violation detection and system call monitoring.
20