arXiv CSopen access

Hunting Vulnerability Variants in AI Infra: Measurement and Reference-Driven Detection

Hunting Vulnerability Variants in AI Infra: Measurement and Reference-Driven Detection Tian Dong∗ , Yanjun Chen† , Shoufeng Zhang∗ , Huaien Zhang∗§ , Yunlong Lyu∗ , Keke Lian† , Dong Zhang† , Shaofeng Li‡ , Hao Chen∗ ∗ The University of Hong Kong † Tencent

arXiv:2605.20051v1 [cs.CR] 19 May 2026

‡ Southeast University

Abstract—AI infra has become a shared execution layer for model training, deployment, and agent orchestration. Because many projects reimplement similar model-centric workflows, a vulnerability disclosed in one repository can recur as a variant in another repository with a related design. Yet the prevalence and detectability of these variants remain poorly understood. This paper presents a measurement study of vulnerability variants in AI infra. Analyzing 688 GitHub repositories and 251 publicly disclosed vulnerabilities, we find that AI infra projects frequently share overlapping functionality and recurrent vulnerable patterns, creating a concrete basis for cross-repository variants. Building on this finding, we study how to automatically identify such variants from known disclosures. We propose I NFRA S COPE, a reference-driven multi-agent framework that extracts transferable vulnerability semantics from known cases and uses them to locate and validate variants in new repositories. Evaluating I NFRA S COPE on 20 real-world AI infra repositories, we uncover over 20 vulnerabilities, including 11 acknowledged cases and 4 cases that have been assigned CVEs so far.

1. Introduction Once an AI infra vulnerability is disclosed, the same failure mode may already exist elsewhere. Modern AI infra is now on the execution path of model post-training [1], serving [2], and agent orchestration [3]. In these workflows, a developer or end user may upload an adapter, point to a remote checkpoint, or configure a tool through a Web UI, CLI, or serving API. The infrastructure layer [4] then fetches external data, deserializes model state, launches commands, and connects models to downstream tools or data sources. This pairing of attacker-controlled inputs with privileged operations makes shared AI infra software a natural attack surface. A public LlamaFactory disclosure (70k+ GitHub stars) illustrates the risk: user-controlled model or adapter paths can reach unsafe model-loading logic and cause remote code execution in repositories that implement related posttraining workflows [5]. As codebase-scale autonomous bug discovery becomes increasingly practical (e.g., Mythos [6]), §. Corresponding author.

each disclosure creates an urgent post-disclosure auditing problem: maintainers must determine whether the same trigger semantics recur in related repositories before attackers scale the search. This audit cannot be solved by code similarity alone. AI infra projects often realize the same functional context through different wrappers, module boundaries, and deployment assumptions. Existing recurring vulnerability detectors work well when transfer follows code reuse [7], [8], [9], patch lineage [8], [10], [11], or stable signatures [12], [13], [14]. These assumptions do not directly cover a disclosed trigger mechanism that reappears in semantically similar forms across independently developed AI infra projects. Repository-scale auditors and Large Language Model (LLM)-assisted analyzers broaden codebase exploration through agentic search [15] or neuro-symbolic reasoning [16]. However, they are usually organized around general bug classes rather than preserving the semantic features of a known reference vulnerability across related repositories. Project-scale studies further show that LLMbased repository analysis remains constrained by shallow interprocedural reasoning [17] and bounded context [18]. These limitations call for an audit procedure that starts from a known vulnerability and searches related AI infra projects for semantically equivalent variants. Motivation. Before building such a procedure, we first examine whether AI infra provides a real empirical basis for cross-repository vulnerability variants. This leads to three research questions: 1. Do public AI infra repositories share enough functional context for cross-repository transfer to be meaningful? 2. Given a public AI infra vulnerability, can its trigger mechanism appear as semantically equivalent variants in repositories with similar functional context? 3. If so, can a public disclosure be turned into a practical auditing procedure that identifies such variants in related repositories? Measurement. We answer the first two questions through a measurement of the public AI infra ecosystem (Section 2). We crawl and analyze 688 AI infra repositories and 251 public vulnerabilities, including 232 disclosures with trigger details sufficient for variant analysis. The results show that

AI infra is rapidly expanding, concentrated in a few major capability families, and repeatedly built from overlapping functional modules. More importantly, public disclosures expose recurring trigger mechanisms under shared functional contexts, not merely isolated project-specific failures. This evidence motivates post-disclosure auditing across related repositories. I NFRA S COPE. We therefore study reference-driven auditing for vulnerability variants (Sections 3 and 4). We call the disclosed case a reference vulnerability and a semantically equivalent case in another repository a vulnerability variant. The core idea is to make manual variant hunting practical by placing tool-augmented LLMs inside a controlled auditing harness. Directly applying agents with conventional software development tools faces three challenges. First, a reference vulnerability gives only partial guidance; an unconstrained agent can spend its budget on irrelevant modules or false candidates. Second, current LLMs have limited context windows [15], [18], making it difficult to sustain a long inspection and preserve relevant memory across sessions. Third, hallucination can produce nonexistent candidates or unsupported conclusions that are hard to filter without manual verification. We propose I NFRA S COPE, a multi-agent framework for identifying vulnerability variants in AI infra. I NFRA S COPE addresses these challenges through a reference-driven auditing harness that coordinates specialized agents for semantic modeling, localization, and verification. First, a semantic modeling agent extracts vulnerability features (e.g., propagation) from the reference and models the target repository as inspection-oriented functional modules with module-level dependencies. Second, an inspection agent uses localizationaware state management to prioritize target modules with matching AI infra functionality and externalize compact audit state, such as inspected files and rejected hypotheses, across bounded sessions. Third, a target-side verification agent combines repository code facts with automated PoC generation to check exploitability before reporting. The workflow assumes existing vulnerabilities and repository code, making post-disclosure auditing practical without private telemetry or privileged deployment access while keeping final claims grounded in repository facts. We implement I NFRA S COPE in over 29K lines of code and evaluate it with 8 reference vulnerabilities and 20 target repositories. In the benchmark, I NFRA S COPE reports 31 candidates, including 24 true positives, and achieves the highest precision and accuracy among the compared tools. It consumes over 7× fewer tokens than Claude Code under the same references while detecting only 4% fewer true positives. Across the benchmark and an OpenClaw case study, I NFRA S COPE uncovers over 20 zero-day vulnerabilities, including 11 acknowledged cases and 4 assigned CVEs so far. Contributions. This paper makes the following contributions: • We conduct a measurement across 688 AI infra repositories and 251 related public vulnerabilities, identifying

recurring overlaps in functionality and vulnerability patterns. • We formalize reference-driven vulnerability variant auditing, where the known vulnerability defines the trigger semantics to preserve and functional context guides localization. • We design and implement I NFRA S COPE, which coordinates three agents on semantics modeling, functionalcontext localization, and target-repository verification. • We evaluate I NFRA S COPE on 8 reference vulnerabilities and 20 target repositories, finding over 20 zero-day vulnerabilities, including 11 acknowledged cases and 4 assigned CVEs so far.

2. AI Infra Ecosystem In this section, we measure recent AI infra ecosystem growth, examine cross-project functional similarity, and test whether public vulnerabilities expose similar trigger mechanisms across related functional contexts. We use AI infra to denote a reusable implementation layer that lets developers build and operate AI systems. For example, an agent orchestration framework (e.g., openclaw) can be integrated with a finetuning framework (e.g., transformers) to build a personalized assistant. AI infra typically includes one or more capabilities such as post-training, serving and inference, data retrieval, agent orchestration, and user interfaces, etc. We exclude generic repositories such as tutorials or prompting guides unless they provide AI-specific reusable infrastructure functionality. To simplify, we manually assign each repository a primary capability label in the measurement and detail the modulerole taxonomy in Section 4. Measurement setup. We first present a summarized measurement setup and refer the reader to Appendix B for more details. We construct a dataset of 688 AI infra repositories through the GitHub Search REST API using topic and keyword queries. For growth, we focus on the 676 repositories created on or after January 1, 2023. Repositorylevel summaries use all 688 repositories. For each repository, we retain the GitHub information needed for the measurement (e.g., description and primary language). We exclude repositories that only mention AI terms but contain auxiliary material or end-user applications rather than reusable infrastructure. Growth tendency. We begin with project counts and programming language usage in the collected AI infra repositories. Figure 1a shows monthly counts of newly created AI infra projects by capability family in our collection. The ecosystem continues to expand across multiple capability families. Specifically, within the 2023+ discovery window, the annual counts of newly created projects increase from 124 in 2023 to 142 in 2024 and 262 in 2025, while the first quarter of 2026 already contributes 148 projects. Notably, the agent orchestration is the largest family in our collection with 373 projects, but serving and inference, RAG and data,

Projects / month

Train Serve RAG Agent Eval/Ops UI/DX

40 20 0 2023

2024

2025

2026

Python TypeScript Rust Go C++ Other

Train Serve RAG Agent Eval/Ops UI/DX 0%

95% 53%

13% 8% 12% 10%

61%

9% 11%

49%

26%

43% 29%

27% 43%

50%

9% 12%

11% 11% 8% 14%

14%

100%

LLaMA Factory verl Open RLHF ms-swift unsloth axolotl TRL

/ 9 6 8 7 10 6 y

ctor

AFa LLaM

0.93 / 7 8 5 10 6

0.91 0.94 / 6 3 7 4

0.92 0.90 0.90 / 5 8 4

0.90 0.87 0.89 0.87 / 6 2

verl nRLHF s-swift nsloth u m Ope

0.90 0.87 0.87 0.86 0.86 / 6 lotl

axo

0.88 0.88 0.88 0.85 0.85 0.83 /

TRL

Figure 1: AI infra measurement summary. (a) monthly project creation by capability family since 2023, (b) primary-language composition within each family, and (c) project-level closeness among the seven analyzed training frameworks: README embedding similarity in the upper triangle and shared training stack features in the lower triangle. and evaluation and observability also contribute substantial volume, with 149, 75, and 63 projects, respectively. Figure 1a also shows a peak in February 2026, when 73 new repositories appear overall and 53 of them fall into agent orchestration and the full monthly cohort spans 72 distinct owners. To test whether OpenClaw imitation explains this peak, we conduct a follow-up OpenClawcentered similarity check on those 53 agent orchestration repositories. In total, we identify 36 OpenClaw-like cases, 9 partially similar cases, and 8 dissimilar cases, suggesting broader family-level reuse. To distinguish creation bursts from repository liveness, we also record each repository’s latest push and study how recently the collected repositories were updated. We find that 428 repositories were last pushed in March 2026, and this concentration is not explained by OpenClaw-similar repositories, which account for only 6 of the 428 cases. Together with the creation trend, the latest push trend indicates that AI infra is both expanding and recently maintained at scale. We next examine whether this active ecosystem is concentrated in a few language and capability families, which would support the feasibility of I NFRA S COPE. Language and type concentration. In Figure 1b, we investigate primary-language distributions across the same capability families using the primary language reported by GitHub. The remaining share is grouped into Other, such as JavaScript in capability families where those languages are present but not separately labeled. As expected, Python is the largest language overall, appearing in 356 of 688 projects. It remains the largest language in agent orchestration, serving and inference, and RAG and data, while TypeScript, Go, Rust, and C++ are also widely used to accelerate programs. Table 1 complements this within-family view with family-level repository and star shares. Agent orchestration also dominates star share, while serving and inference and RAG and data form a second tier by project volume. The platform and training family remains smaller by project count but still captures a non-trivial star share, indicating influence concentrated in a smaller set of repositories. The language and family shares show concentration in several major capability families, but family labels alone do not establish transferable functionality. We therefore inspect

TABLE 1: Repository and star shares of AI infra capability families in the dataset. AI infra Type Agent orchestration Serving and inference RAG and data Evaluation and observability Platform and training UI and workflows

Projects

Project Share

Stars

Star Share

373 149 75 63 21 7

54.2% 21.7% 10.9% 9.2% 3.1% 1.0%

3,114,853 511,233 699,624 285,164 437,192 10,400

61.6% 10.1% 13.8% 5.6% 8.6% 0.2%

whether related repositories expose similar implementation features. To further examine functional similarity between projects, we take a closer look at training frameworks, a type of AI infra with repeated workflows and a common programming language (Python). Functionality closeness. We select the seven representative training frameworks [1], [19], which collectively exceed 200k GitHub stars as of March 2026. We represent each framework with 12 binary structural and workflow features such as distributed training and inference augmentation. We also compute README-level semantic similarity using normalized embeddings produced by bge-large-en-v1. 5 [20]. Figure 1c shows two complementary measurements. Entries in the upper triangle report README embedding similarity, whereas entries in the lower triangle count how many of the 12 structural and workflow features are shared by each framework pair. The seven frameworks are not near-duplicate projects, but their README semantics and shared workflow features still form visible similarity clusters. The median off-diagonal embedding similarity is 0.88 (IQR 0.04), and the strongest pair reaches 0.94 for verl and OpenRLHF, followed by 0.93 for verl and LLaMA-Factory and 0.92 for LLaMA-Factory and ms-swift. Finding 1: AI infra is a rapidly expanding, Python-centered ecosystem with overlapping functional modules. This overlap supplies the functional substrate required for similarity-guided variant transfer across modules and projects.

Trigger-level similarity in public disclosures. The remaining question is whether shared AI infra functionality can lead to vulnerability variants, which share a similar functional context, trigger scenario, and vulnerability family.

11

5

11

3

3 12

4 6

4 2

7

5

2

2

4

10 2

2

2

3

2 2

SQ inj. L Cm inj.d. Y inte AML rp.

F SSR

R/WFile

3 P traath v. A byputh . Cod inj. e

Vulnerability variants are not limited to directly copied flaws; they preserve the semantics of the reference vulnerability despite code changes such as refactoring and API migration across AI infra projects. They can exist across different modules in the same AI infra project or across similar repositories. For instance, more than 17 SQL injection variants appear in different database modules of the agent orchestration project llama_index (see Figure 6). We next present a cross-repository example of vulnerability variants as shown in Figure 3. In CVE-2025-53002, the affected LLaMA-Factory code receives a user-uploaded adapter path (i.e., source). The path can point to malicious serialized data from an attacker-controlled remote source (e.g., a Hugging Face model). Then, the vulnerable code directly deserializes the adapter with torch.load (i.e., sink), which can lead to remote code execution (RCE) if the attacker crafts a malicious payload. The same question arises for ms-swift, which implements a comparable post-training capability. The target contains related UI logic and model-loading modules, but the propagation path differs. In ms-swift, model loading relies not on direct function calls but on command execution through swift -params in llm_train.py. A call graph-based approach that searches for a direct relation from the source to sink would not model this command-mediated propagation and can miss the variant. I NFRA S COPE found the vulnerability variant GHSA****-****-2cf3 by transferring the trigger semantics of CVE-2025-53002. This case starts from a WebUI or training entry and propagates the user-provided adapter model path into the executed command. The command executed through os.system eventually reaches torch.load without a secure deserialization check [5]. Although the two vulnerabilities expose the same deserialization pattern (a UItriggered training path reaches tuning and model-loading logic that deserializes adapter state with torch.load), they manifest in different affected modules due to distinct implementations.

3

2

11 2

6

4

5

2

3.1. Motivating Example

5

4

13 2

7

Projects

16

R exhes. . U denssafe er.

Functional context

Serving API Model load Eval / observ. File transfer RAG ingest UI workflow Auth / session Agent tools Config / runtime Network fetch Training

1

Vulnerability family

Figure 2: Similar vulnerability patterns in AI infra disclosures. Bubbles aggregate functional-context and vulnerability-family pairs supported by public trigger details. Number and area encode deduplicated vulnerability count, and color encodes affected projects. We curate a dataset of publicly disclosed vulnerabilities from public advisory databases [21], [22], [23], vendor writeups and huntr reports [24]. The main dataset covers disclosures since January 1, 2024 and keeps only records that map back to reusable AI infra repositories or packages. To avoid treating sparse metadata as support, we require public trigger details to investigate whether similar functionality carries similar vulnerability mechanisms, not merely whether projects share broad CWE labels. After manually deduplicating multiple sources for the same disclosure and excluding non-AI infra records, the main dataset contains 597 source records covering 251 public vulnerabilities across 18 projects. Among them, 232 have public trigger details. Figure 2 summarizes these patterns with public trigger details after grouping rows by context and vulnerability family. The strongest cross-project patterns include serving API resource exhaustion (16 deduplicated vulnerabilities across 4 projects), access-control bypasses (13 across 4 projects), attacker-controlled URL fetches leading to SSRF (11 across 7 projects), and model-loading deserialization bugs (9 across 5 projects). Finding 2: Public AI infra disclosures reveal similar trigger mechanisms under shared functional contexts across projects, rather than only isolated project-specific failures. This evidence justifies functionality-guided localization, while reportable variants still require semantic consistency in the target code.

3. Vulnerability Variants in AI Infra In this section, we first show a motivating example of vulnerability variants across two training repositories, then derive the design challenges and finally formalize the detection problem.

3.2. Challenges This example motivates our reference-driven formulation: the transferred object is trigger semantics, not a hosting boundary, exact call graph, or coarse CWE. Therefore, we require three transfer cues. Similar entry identifies user-facing training functionality rather than identical code. Pattern generalization abstracts the disclosed bug into an attacker-controlled asset path, cross-module propagation, and missing guard. Similar functionality localizes target modules that implement similar behavior. LLM-assisted vulnerability analyzers either augment static analysis with taint specifications, code property slices, or constraints [16], [25], [26], or use repository-level agents to search for vulnerable flows [15], [27], [28]. These formulations are usually organized around general vulnerability classes or local sourcesink reasoning in the target, rather than preserving the trigger

UI Module (src/llamafactory/webui) engine.py 28: class Engine: 35: self.runner=Runner(..) 36: self.chatter=WebChatModel (..)

chatter.py 80: class WebChatModel(ChatModel): 81: def__init__(..): 87: super().__init__()

Chat Module

(src/llamafactory/chat)

chat_model.py 42: class ChatModel: 50: def__init__(..): 53: self.engine=HuggingfaceEngine(model_args, ..)

loader.py

133: def load_model(..,model_args,..): 195: if model_args.adapter_name_or_path is not None: 196: vhead_path=model_args.adapter_name_or_path[-1] 200: vhead_params=load_valuehead_params(.., model_args) model_utils/valuehead.py

runner.py

55: class Runner: 128: def _parse_train_args(..): 184: if get("top.checkpoint_path"): 186: args["adapter_name_or_path"] =",".join( [get_save_dir(model_name, finetuning_type, adapter) for adapter in get("top.checkpoint_path")])

Reference: CVE-2025-53002 Target: GHSA-****-****-2cf3

UI Module (swift/ui)

Model Module (src/llamafactory/model)

Similar Entry

app.py 71: LLMTrain.build_ui(LLMTrain)

llm_train/llm_train.py 242: def do_build_ui(..): 297: submit.click(cls.train_local,..) 491: def train_local(cls, *args): 492: run_command, sft_args, other_kwargs=cls.train(*args) 426: if more_params_cmd!='': 427: params+=f'{more_params_cmd.strip()} ' 463: run_command=f'{cuda_param}{ddp_param}{envs} nohup swift {cmd}{params} > {log_file} 2>&1 &' 508: os.system(run_command)

hf_engine.py 44: class HuggingfacxeEngine(BaseEngine): 45: def__init__(model_args, ..): 59: self.model=load_model(self.tokenizer, model_args, ..)

Pattern Generalize Tuning Module swift/train/sft.py 105: self.model=self.prepare_model( ..)

swift/train/tuner.py 345: classTunerMixin: 348: def prepare_model(cls, ..) 353: if args.is_adapter: 359: if args.resume_from_checkpoint: 363: tuner = Swift 371: model=tuner.prepare_model(args, model)

33: def load_valuehead_params(.., model_args, ..): 38: kwargs= {.., "cache_dir": model_args.cache_dir, ..} 50: try: 51: vhead_file = cached_file(filename=V_HEAD_WEIGHTS_NAME, **kwargs) 52: return torch.load(vhead_file, map_location="cpu")

Similar Functionality swift/tuners/base.py 32: class SwiftModel(nn.Module): 242: def load_state_file(...): ... 257: elif os.path.exists(os.path.join(path, WEIGHTS_NAME)): 258: filename=os.path.join(path, WEIGHTS_NAME) 259: return torch.load(filename, map_location=device) 297: def from_pretrained(cls,..): 356: for _name in adapter_name if isinstance(adapter_name, ..)..: 365: state_dict=cls.load_state_file(sub_folder)

Figure 3: Cross-repository motivating example for vulnerability variants in AI infra found by I NFRA S COPE. A disclosed LLaMA-Factory RCE (CVE-2025-53002) and an ms-swift advisory (GHSA-****-****-2cf3) share a similar source in the UI module, where a user-controlled adapter path reaches the unsafe loading sink torch.load through parameter passing and command invocation (dashed arrow), respectively. semantics of a disclosed reference case. AI infra widens this gap because wrappers, indirection layers, and naming variation separate functional similarity from local syntactic similarity. Therefore, I NFRA S COPE faces three challenges. C1: How to model the semantics of the reference vulnerability and the reference and target repositories? Because variants may preserve a trigger mechanism while moving across modules and implementations, the system needs semantics that capture functional modules and security-relevant components while abstracting away from irrelevant details. Previous recurring-vulnerability detectors often rely on code-reuse signatures [7], [8], [9], taint signatures [12], patch-derived signatures [8], [10], [11], or semantic-equivalence signatures [11], [29], so their matching assumptions may not capture the specific trigger condition and semantics of a disclosed vulnerability. Direct use of LLMs for semantic modeling of project-level code and vulnerabilities can also fail: recent project-scale studies report shallow interprocedural reasoning and misidentified source-sink pairs [17], and long-context models can underuse critical context buried in the prompt [18]. Solution: To address this challenge, we design a semantic modeling agent. It uses LLM-based code understanding to construct functional modules and static analysis to model module-level call graphs. On the repository side, this approach identifies relevant user roles and application scenarios that narrow the search space (e.g., training framework). On the vulnerability side, it infers deployment and input boundary assumptions from the reference case (e.g., loading from an untrusted source) and maps the vulnerability to a guiding semantics that captures the trigger conditions and propagation patterns.

C2: How should candidate inspection be localized under a bounded model context and inspection memory? LLM agents can interleave reasoning with tool use [3], which makes them attractive for repository-level auditing. However, long inspections accumulate code snippets, search results, and failed hypotheses faster than the active context can retain them. Existing repository auditors encounter similar limitations: context size, hallucination risk, and token cost must be managed by memory and on-demand exploration [15]. Moreover, generic context-management techniques, such as virtual-context memory [30], reflective episodic memory [31], and prompt compression [32], can preserve compact histories across long interactions. These techniques are insufficient for reference-driven variant detection. First, a generic summary of the inspection history may retain the agent’s conclusion while dropping failed hypotheses about sources and sinks that are needed for auditability. Second, a larger retained context still does not determine what to inspect next. Models can underuse critical context buried in long prompts [18], while variant inspection must spend budget according to the disclosed trigger semantics. Solution: To address this challenge, we design a localization-aware state management scheme that keeps the agent’s search aligned with the reference functionality under bounded context. The key idea is to isolate critical history (e.g., searched modules and hypotheses) from the session context as cross-session inspection memory so that each session restart or context compression can resume without losing inspection-required information. The inspection agent externalizes compact audit state, such as completed files, so later turns can avoid replaying broad repository searches. C3: How should final findings remain auditable given

potential LLM hallucinations? LLM hallucination is a known failure mode in neural text generation [33], and it is especially costly in vulnerability reporting: an invented source location or exploit precondition can turn a benign path into a misleading disclosure. Because manual verification is time-consuming, a natural baseline is to introduce an LLM judge to reason over the reported candidates. However, an LLM judge may be biased by the candidate report itself, and hallucinated claims can be hard to distinguish without verification against the target repository. Solution: To address this challenge, we implement a dynamic verification loop for candidate findings through automated PoC generation with a coding agent. The verifier reconstructs the path from attacker-controlled input to the sensitive operation and checks whether the missing guard and preconditions preserve the reference semantics. When execution is feasible, a sandboxed PoC records whether the exploit reaches the candidate’s code path. Otherwise, unresolved conditions remain explicit and keep the conclusion conservative. The final report contains static code facts and verification logs needed to reproduce the finding.

Definition 3 (Vulnerability variant). Given a target codebase S , a variant v of the reference vulnerability v ⋆ is a vulnerability in S whose source and sink types are semantically aligned with v ⋆ and whose witness path π(v) preserves the trigger semantics of v ⋆ while differing in implementation details such as API migration. Our task can be formalized as follows: given a reference vulnerability v ⋆ with semantics ΣV (v ⋆ ) in a reference codebase S ⋆ , detect variants v in a target repository S such that their vulnerability semantics are aligned with the reference, ΣV (v ⋆ ) ∼ ΣV (v), and their repository semantics are similar, i.e., ΣR (S) ∼ ΣR (S ⋆ ).

4. I NFRA S COPE Design In this section, we present I NFRA S COPE, a referencedriven auditing harness for vulnerability variants in AI infra, implemented as a controlled orchestration layer around LLM agents.

4.1. Overview 3.3. Problem Statement We consider repository code and use codebase and repository interchangeably throughout the paper. We treat two commits of the same project as distinct repository revisions when their relevant semantics differ. We next define repository semantics, reference vulnerability semantics, and vulnerability variants. Definition 1 (Repository semantics). We model the semantics of a repository S as: ΣR (S) = ⟨ρ(S), M(S), ϕ(M(S)), GM (S)⟩.

(1)

Here ρ(S) denotes a compact summary of the repository for comparison across projects. M(S) denotes functional modules based on a role taxonomy for AI infra, Vrole . Each module m has a descriptor ϕ(m) summarizing its assigned role and supporting code facts. The graph GM (S) records dependencies across modules. This abstraction follows how human auditors reason about a codebase: the repository summary supports comparison across projects, while module functionality and calling structure support bounded inspection. Definition 2 (Reference vulnerability semantics). Let v ⋆ be a reference vulnerability discovered in a reference codebase S ⋆ , described by a witness call chain π(v ⋆ ) from source to sink. We abstract v ⋆ into vulnerability semantics with a fixed schema: ΣV (v ⋆ ) = ⟨ψ(π(v ⋆ )), A(S ⋆ , π(v ⋆ ))⟩,

(2)

where ψ encodes semantic features of the witness path π(v ⋆ ), and A(S ⋆ , π(v ⋆ )) represents the affected modules along π(v ⋆ ).

The harness coordinates three specialized agents: a semantic modeling agent, an inspection agent, and a verification agent. For clarity, we present these agents in a workflow with three phases (Figure 4), but the actual implementation can be intertwined and iterative. Algorithm 1 summarizes this workflow. Given a reference vulnerability v ⋆ and a target codebase S at commit c, I NFRA S COPE first models the vulnerability semantics ΣV (v ⋆ ) and the repository semantics ΣR (S, c) (Phase 1). ΣV (v ⋆ ) provides downstream inspection context, such as the trigger condition and affected modules, while ΣR (S, c) combines a repository summary ρ(S, c) for target selection across repositories with module structure for downstream localization. In Phase 2, with the prepared vulnerability and repository semantics, I NFRA S COPE selects the most similar targets based on the repository semantics to avoid unnecessary inspection, and then inspects the selected targets in priority order. The inspection agent, provided with a set of tools that human experts would use for code inspection, iteratively searches for modules relevant to the reference vulnerability based on its semantics. The inspection state is maintained in a local memory Z so that coverage, candidate state, and stopping conditions remain explicit when the live context is compacted across bounded iterations. The inspection stops according to the configured iteration budget. Finally, in Phase 3, the verification agent evaluates each candidate against the target repository under the assumptions required by the reference vulnerability. To enable dynamic verification, the verification agent runs inside a coding agent that sets up a sandbox environment and generates a PoC to confirm whether the candidate can reach the same vulnerable path as the reference case. The verifier then generates a conclusion and verification evidence (e.g., dynamic verification logs).

Phase 1: Semantics Modeling

Existing CVE

detected

Phase 2: Candidate Inspection Reference RS Reference VS

Reference Repository

Search Target Repo

Semantics Agent

Payload Callchain Description

Target User Semantic Module Call Graph

Repository Vulnerability Semantics (VS) Semantics (RS)

Top Similar Candidates Priority-1 Priority-2 …

Relevant Modules

Context Compression

Inspecion Agent Log

Toolset       

Use

read_file search list_folder get_function analyse_AST SAST_tool …

Verification Agent Generate

Load Candidates

Memory      

Phase 3: Verification Loop

Checked Modules Checked Files Progress summary Reasoning pitfalls Existing Candidates ….

     

Bug type Description Source API/function Sink API/function Callchain Evidence

    

Read

Feedback

Exploits & PoC

Apply

Bug Report

Yes

Title Severity Assessment PoC Affected Modules Impact

Software Configuration Set up Verification Sandbox

Triggerable? No Discard candidate

Figure 4: Overview of I NFRA S COPE. The semantic agent derives vulnerability and repository semantics. Then, the inspection agent uses these semantics to prioritize target inspection under memory and context budgets. Finally, the verification agent confirms candidates with static code facts and sandbox validation before reporting findings.

4.2. Semantics Modeling Agents perform poorly when the inspection context is underspecified. Although advanced agents can summarize a codebase automatically, they can miss critical components (e.g., module call graphs) that support downstream variant detection. We extract semantics for the reference vulnerability and target repository before inspection, using them as the map for downstream localization. Repository Semantics. Following Definition 1, the semantics of a repository S include a repository semantic summary ρ(S), modules M(S), functionality descriptors ϕ(m) for each module m, and a module call graph GM (S). Directly prompting an LLM over every file requires many tokens and is unstable across repositories. Furthermore, without a unified category scheme, the generated module names can also differ across projects, which hinders repository comparison. We define a role taxonomy with two levels of categories for common AI infra functionality. This taxonomy constrains both module assignment and similarity assessment (see Table 10 in Appendix C.4 for details). The semantics agent assigns each file to one or more module roles using this taxonomy. It first relies on path names, package structure, and local context, then reads source snippets only when the assignment is ambiguous. For example, post-training functionality can span modules including optimizer configuration, adapter loading, checkpointing, and model loading paths. These cases require source evidence rather than directory names alone. During assignment, the agent stores concise descriptive features that help the downstream inspection agent decide whether a module should receive inspection budget for variants. This process produces the modules M(S) and the functionality descriptors {ϕ(m)}m∈M(S) , each of which includes: ϕ(m) = ⟨ℓ(m), files(m), funcs(m), deps(m)⟩ ,

(3)

where files(m), funcs(m), and deps(m) denote the relevant files, important functions and classes, and dependencies,

respectively. To build the dependency graph across modules GM (S), we use an existing Static Application Security Testing (SAST) framework (e.g., CodeQL [34]) to extract call relations and then project them onto the assigned module descriptors {ϕ(m)}m∈M(S) . The module call relations define how far inspection may expand from the matched functionality. In addition, we extract a compact repository summary ρ(S) through targeted semantic abstraction of the repository description, application scenario, target user, and key external dependencies. This summary is used to support candidate target selection by similarity, whereas the downstream inspection agent localizes variants through M(S), ϕ(M(S)), and GM (S). We elaborate these repository and module similarity measures in the evaluation (Section 5). Vulnerability Semantics. To enable transfer across projects and modules, vulnerability semantics must be narrower than a CWE class but more portable than signatures tied to one project. They must also make explicit the assumptions under which the reference vulnerability is exploitable. Given the call chain, payload, and affected codebase of the reference vulnerability v ⋆ , the agent generates ψ(π(v ⋆ )) from source to sink, including the trigger condition, propagation constraints, exploitable scenario, missing guard, and affected trust boundary. ψ(π(v ⋆ )) is then used by the inspection agent to identify candidate variants and check reachability. During this process, the agent reads relevant code snippets from S ⋆ to recover affected modules A(S ⋆ , π(v ⋆ )). These affected modules, together with target repository semantics, feed the priority-guided inspection procedure.

4.3. Semantics-aware Inspection Before inspection, I NFRA S COPE filters and ranks candidate repositories by similarity between their semantics and the reference repository semantics, retaining targets (S, c) for which ΣR (S, c) ∼ ΣR (S ⋆ ). In practice, we compute this similarity using information derived from both the repository

summary ρ(·) and the module structure in M(·), ϕ(M(·)), and GM (·). For each target revision (S, c), the inspection agent uses module priority to order search and inspection memory to preserve progress. Priority Scheme. The priority scheme assumes that vulnerability variants are most likely to appear in target modules that implement functionality similar to the modules affected in the reference. It uses three priority levels over modules. Let A(S ⋆ , π(v ⋆ )) be the affected modules derived from the vulnerability semantics and M(S, c) be the target modules from the repository semantics. Priority 1 includes modules that implement functionality affected in the reference, including modules matching A(S ⋆ , π(v ⋆ )). It may also promote a target module to priority 1 when the embedding similarity between its descriptor text and an affected module name exceeds a threshold τM :

Algorithm 1: Reference-driven variant inspection. Input: Reference vulnerability v ⋆ with reference codebase S ⋆ and target revision (S, c). Output: Findings verified against the target repository F . ⋆ 1 ΣR (S ) ← E XTRACT R EPOSITORY S EMANTICS(S ⋆ ) ⋆ 2 ΣV (v ) ← E XTRACT V ULNERABILITY S EMAN TICS(v ⋆ , ΣR (S ⋆ )) 3 ΣR (S, c) ← E XTRACT R EPOSITORY S EMANTICS(S, c) 4 (P1 , P2 , P3 ) ← P RIORITIZE M ODULES(ΣR (S, c), ΣV (v ⋆ )) 5 Z ← I NSPECTION M EM ORY (ΣR (S, c), ΣV (v ⋆ ), P1 , P2 , P3 ) 6 C ← ∅, i ← 0 7 while i < maximum iterations do P1 = Pname ∪ Pemb , where ⋆ ⋆ 8 i←i+1 Pname = {m ∈ M(S, c) | m ∈ A(S , π(v ))}, 9 Build context from ΣV (v ⋆ ), (P1 , P2 , P3 ), and Pemb = {m ∈ M(S, c) | max simM (ϕ(m), a) ≥ τM }. Z a∈A(S ⋆ ,π(v ⋆ )) 10 Run one bounded inspection turn over (S, c) A module belongs to priority 2 when it directly calls a 11 Update candidates C ← C ∪ {⟨loc, π, Estatic ⟩} priority 1 module or is directly called by one: and inspection memory Z ⋆ P2 = {m ∈ / P1 | callsin (m)∩P1 ̸= ∅∨callsout (m)∩P1 ̸= ∅}. 12 F ← V ERIFY C ANDIDATES (C, ΣV (v ), S, c) 13 return F Direct caller and callee expansion captures wrapper and orchestration drift while keeping the search bounded. All remaining modules become priority 3. The agent starts from files of P1 for potential sinks and sources, then P2 , then while preserving search continuity, I NFRA S COPE applies priority 3 modules. compaction before each query when the predicted request Memory Management. To keep inspection state explicit budget approaches the context window. Each completed and reuse history from previous inspections, I NFRA S COPE iteration is compressed into a compact reasoning summary maintains two memory layers. The first is a local inspection that retains only inspection-relevant records, such as failures memory Z including the status of each file within the current and reusable shared memory hits, while Z remains the inspection scope, assignments from files to modules, module source of truth for coverage and candidate state. priorities, completion reasons, potential candidates, and the critical scope stopping boundary for (S, c). The second is 4.4. Exploitability Verification a shared public memory that stores compact observations distilled from previous inspection runs over the same target The inspection agent can report a false finding due to repositories, such as lightweight data flow summaries, that hallucination (e.g., a nonexistent path), so the verification can avoid repeated reasoning or tool usage. This shared agent checks whether each candidate preserves the reference layer can help later inspections avoid recomputation and pattern under code facts from the target repository. In our redundant broad exploration. prototype, this stage is implemented as a reusable verificaIterative Inspection. The inspection is organized as a tion skill [35] supported by the coding agent in Section 5. sequence of bounded agent iterations rather than a fixed The skill first performs static claim checking under the single pass over pending files. At iteration t, the agent assumptions recovered from the reference vulnerability and constructs the context from ΣV (v ⋆ ), the priority tiers, and returns one of four conclusions: the current state in Z , including the progress summary, the • Exploitable: the target code contains a weakness that remaining files in the configured critical scope, previously follows the reference pattern and supports a concrete scanned files, and already recorded candidate reports. The attack scenario. inspection agent then executes one bounded tool call turn • Conditionally exploitable: the weakness that follows the over the target codebase and updates Z explicitly through reference pattern is supported only under explicit input candidate reporting and file completion actions. A turn preconditions. ends when the inspection agent emits no further tool calls, • Library risk: a risky library dependency exists, but the explicitly indicates completion, or is stopped by the context current target does not expose an exploitable path. or turn budget. The outer loop proceeds until the configured • Non-exploitable: the candidate lacks attacker-controlled iteration bound and critical scope stopping policy are satissource, reachability, semantic alignment, or is blocked fied. To keep enduring inspections within the model context

by effective protections. When the initial conclusion indicates exploitability and the target can be safely configured, the verifier generates and tests a minimal PoC in an isolated container, with multiple retries to resolve potential errors (e.g., missing dependencies).

TABLE 2: Statistics of the eight reference vulnerabilities used in the vulnerability profile accuracy evaluation. The call chain length counts source-to-sink entries. Domain

Repository

CVE ID

Vuln. Type

Sim.

Training

NeMo LlamaFactory Megatron-LM

CVE-2025-23361 CVE-2025-53002 CVE-2025-23348

5.29/3.00 8.36/2.81 9.49/2.01

3 7 6

Cmd. Inj. Deser. Code Inj.

0.9377 0.9095 0.8791

Agent

ms-agent AutoGPT langchain-community llama_index AutoGPT

CVE-2026-2256 CVE-2025-22603 CVE-2025-6984 CVE-2025-1793 CVE-2025-1040

8.46/2.43 5.58/2.57 4.61/1.99 4.31/1.70 4.17/1.62

3 4 3 2 2

Cmd. Inj. SSRF XXE SQL Inj. SSTI

0.8849 0.8815 0.9518 0.8466 0.8063

5. Evaluation

In/Out Tok. (K)

Chain

In this section, we evaluate I NFRA S COPE on popular real-world AI infra and investigate the following research questions: • RQ1: How accurately do the repository and vulnerability semantics align with the human labeling? • RQ2: How does I NFRA S COPE compare with baseline methods in terms of detection performance and efficiency? • RQ3: How well does I NFRA S COPE detect vulnerability variants in target repositories, and what are the main sources of false positives and false negatives? • RQ4: How does I NFRA S COPE perform on agent projects with multiple similar modules?

TABLE 3: Target repositories as our evaluation benchmark.

5.1. Setup

the C ODE QL database, run the inspection procedure, and validate all reported findings using the validation protocol. Reference Vulnerabilities & Repository Candidates. Table 3 shows the target repository benchmark, and Table 2 lists the eight public reference vulnerabilities used in the evaluation. We select the reference vulnerabilities and target repositories from the dataset in Section 2 using three criteria: 1) the repository has high stars or active maintenance, 2) the disclosed data provides the call chain, affected version, and payload, and 3) the project has similar counterparts for variant detection. After manual analysis, we select two domains, Agent & RAG, and Training & Inference, because they contain multiple functionally similar repositories and representative vulnerability families. We focus on injection (e.g., SQL and command injection), SSRF, XXE, SSTI, and deserialization vulnerabilities because they are common in the dataset and can be evaluated in a sandboxed setting. Baselines. We compare against two baseline detectors: Claude Code, a fully agentic repository auditor, and Vulnhalla, a SAST pipeline augmented with LLMs [38], [39]. For Claude Code, the prompt includes the vulnerability type, source-to-sink call chain, target repository, and code inspection tools (e.g., CodeQL), and asks it to search for semantically similar vulnerabilities. For Vulnhalla, we create CodeQL queries for each reference vulnerability family and use LLM reasoning to verify and filter CodeQL reports. Metrics. For semantic alignment, we compare generated vulnerability and repository semantics against human annotations. For vulnerability variant detection, we merge candidates reported by all tools into a common candidate dataset and manually label each tool’s decision. We compare I NFRA S COPE with the baselines using precision, recall, accuracy, and token usage.

LLM & Implementation. Throughout the evaluation, we adopt the open-source D EEP S EEK-V3.2 [36] to ensure reproducibility. We run local inference with vllm 0.18.0 [2] with decoding temperature 0.1 and top-p 0.9. We use the official D EEP S EEK API as a fallback when local inference fails. We implement I NFRA S COPE as a Python prototype with 29K lines. It integrates CodeQL 2.23.7 [34] for call graph extraction and agentic SAST queries, and Claude Code (CC) 2.1.86 [37], powered by D EEP S EEK-V3.2, as the coding agent that executes reusable verification skills in a controlled workspace (i.e., in the verification stage). To select candidate repositories for inspection, we compute an overall similarity score as the equally weighted average of five components: semantic description similarity, application similarity, user similarity, module Jaccard overlap, and dependency Jaccard overlap. For the first three components, we compare the corresponding repository profile texts with a local embedding model. For the latter two, we compare the module sets and dependency sets derived from the repository semantics. For each reference vulnerability, we rank all profiled target revisions by this score, keep revisions whose overall similarity is at least 0.5, and, when fewer than three revisions satisfy the threshold, supplement the scan set to the top 5 most similar revisions. If S ⋆ and S differ only in commit revision, we include one additional target for inspection. For module promotion guided by priority within each selected target, we use the default module embedding similarity threshold τM = 0.8 throughout evaluation. We set the maximum inspection iteration to 3 per repository. For each target repository, we extract repository semantics, build

Type

Applications

Commit

Stars (K)

LoC (K)

Modules

In/Out Tok. (K)

Agent & RAG

AutoGPT langflow dify langchain ragflow autogen llama_index mem0 langchain-community ms-agent

42b9facd e9d1c2fb 4461df1b f2dab562 fe4852cb 13e144e5 74e5113c 84687fc3 39be54ca 3d371dc7

183.76 147.35 139.15 134.93 78.98 57.43 48.94 54.04 0.26 4.20

168.58 228.05 596.61 155.14 190.00 79.92 255.02 58.28 214.57 13.22

24 32 32 28 22 23 31 17 14 12

216.97/15.75 472.56/32.35 693.51/49.46 164.96/12.69 206.60/14.59 85.57/6.83 536.24/52.71 60.76/4.31 90.01/7.00 14.99/1.41

Training & Inference

LlamaFactory NeMo Megatron-LM ms-swift BentoML IsaacLab IsaacSim GPT-SoVITS Megatron-Bridge Model-Optimizer

767b344f b515e732 e3ae3511 1a801723 2d289a3f 90af2be2 47d886f2 11aa78bd ab225da7 b660d39a

70.34 17.10 16.04 13.90 8.56 7.00 3.10 56.70 0.59 2.60

28.91 416.05 122.77 64.92 52.25 111.72 254.54 39.52 45.64 60.19

21 35 24 20 15 18 17 11 14 14

31.40/2.48 212.86/13.71 56.49/4.23 35.12/2.66 44.07/3.25 173.68/12.75 288.84/20.96 25.85/1.80 36.44/2.59 39.81/2.45

5.2. RQ1: Semantics Alignment We first examine whether the semantics align with human annotations and accurately represent the repositories and vulnerabilities. Annotation Protocol. We first test whether generated repository and vulnerability semantics match expert interpretation. For vulnerability semantics, two authors summarize each reference vulnerability using the same source material, and a third author resolves the final human annotation. For repository semantics, the same protocol is used for repository summaries in text form. The two authors also inspect module assignments and correct the assigned roles using the predefined module role taxonomy. The third author then resolves disagreements over the union of corrected assignments. Vulnerability Semantics. We first examine whether the extracted vulnerability semantics accurately capture the trigger semantics of the reference cases. As shown in Table 2, the similarity scores between the descriptions annotated by humans and the generated semantics are consistently above 0.8, indicating that the generated semantics are largely aligned with human understanding after verification. In particular, the generated semantics correctly characterize key vulnerability aspects, including vulnerability type, propagation constraints, critical sinks, and potential consequences. The scores do not approach 1.0 mainly because the semantic descriptions generated by D EEP S EEK are more concise than the human annotations, even when detailed contextual information is provided in the prompt. Overall, the high semantic consistency suggests that the extracted vulnerability semantics can provide reliable guidance for subsequent variant inspection. Repository semantics. Table 3 reports the 20 target repositories used in our benchmark for vulnerability variant detection, including repository popularity, code scale at the profiled commit, the number of semantic modules extracted by our pipeline, and the reconstructed full token volume for generating the latest codebase semantics. In total, codebase semantics generation over these repositories accounts for 3.49M input tokens and 263.98K output tokens after counting both the repository summary stage and all module analysis batches. This is approximately 0.63 USD under the official D EEP S EEK API pricing. A key takeaway is that the recorded profile cost is not determined solely by repository size. Instead, it is jointly shaped by the difficulty of identifying stable module boundaries, the diversity of subsystems within the repository, and the amount of analysis needed before the module assignments converge. As a result, repositories with comparable or smaller code volume may still incur higher token cost if their semantic structure is harder to separate cleanly. For instance, llama_index has 255.02 KLoC and 31 modules, and its semantics generation consumes 536.24K input tokens. By contrast, NeMo has 416.05 KLoC and 35 modules, but consumes only 212.86K input tokens. Likewise, langflow has 228.05 KLoC and 32 modules and consumes 472.56K input tokens, whereas IsaacSim has 254.54 KLoC and 17 modules but consumes

only 288.84K input tokens. These cases indicate that the dominant cost factor is semantic ambiguity rather than raw code volume. We next examine whether the generated semantics are actually aligned with human perception. Figure 5 presents the alignment with modules assigned by humans on the left and the mutual similarity between the repository semantics on the right. Overall, Figure 5(a) shows that repository semantics are aligned with human judgment at the semantic level, while exact module boundaries remain harder to reproduce. Across all 20 repositories, the generated module descriptions reach an average BGE similarity of 77.46% against the human annotations. The average coarse module match is 67.31%, and the exact module match is 51.63%. For the balanced subset shown in the left panel, the coarse and exact match rates increase to 86.81% and 69.88%, respectively. This indicates that the generated semantics usually capture the main functional role of a component, even when the exact fine module role differs from the human annotation. The variation mainly comes from how clearly a repository separates functional responsibilities. For instance, IsaacSim, autogen, and Model-Optimizer have high alignment because their files are organized around stable and recognizable roles. IsaacSim is dominated by UI workflow, dataset construction, and template components, and reaches 99.41% coarse match and 94.45% exact match. Autogen similarly exposes clear agent orchestration, integration, and UI components, giving 94.95% coarse match and 91.21% exact match. Model-Optimizer is also easy to annotate because most files fall into inference runtime, model export, model definition, and evaluation roles, resulting in 93.81% coarse match and 84.54% exact match. The lower alignment cases show where repository semantics are still useful but human and generated boundaries diverge. LlamaIndex has a high description similarity of 77.51%, but only 14.29% coarse match and 12.96% exact match. Its codebase contains a large number of supportive modules for agent such as vector storage and retrieval components. Human annotations tend to group many of these files by their RAG role, while the generated semantics split or merge them according to package-level structure. Megatron-Bridge shows a similar pattern. Although its generated descriptions remain semantically close to the human annotations, its exact match is only 9.9% because training configuration, model conversion, dataset building, and checkpoint handling are tightly interleaved in the repository. These cases suggest that low agreement at the file level does not necessarily mean the semantic description is wrong. It often reflects ambiguity in module boundaries. In Figure 5(b) and Figure 5(c), we measure relatedness among repositories. The four Agent & RAG repositories occupy the upper rows and left columns, while the four Training & Inference repositories occupy the lower rows and right columns. The average descriptive similarity is 0.74, whereas the average module Jaccard similarity is 0.35. Compared with the average, the blocks in the upper left and lower right have higher values. This difference is expected because repositories can implement similar workflows using different

Embedding Similarity

100

Coarse Match

Exact Match AutoGPT LlamaIndex LangChain Community MS-Agent LLaMA-Factory IsaacSim Megatron-LM GPT-SoVITS

Score (%)

80 60 40 20 0 aut

n oge

T

oGP

Aut

dify

flow

rag

m

cSi

Isaa

y LM delctor Moimizer atronaFa g t m e p a l M O L

(a) Alignment with human-assigned modules.

0.77 0.72 0.72 0.74 0.70 0.69 0.70 0.65 0.77 0.82 0.80 0.76 0.76 0.77 0.73 0.60 0.77 0.75 0.82 0.73 0.76 0.73 0.67 0.62 0.82 0.78 0.79 0.76 0.76 0.75 0.76 0.70 0.76 0.80 0.79 0.76 0.78 0.87 0.83 0.66 0.71 0.69 0.70 0.66 0.69 0.67 0.74 0.87 0.75 0.75 0.76 0.74 0.78 0.81 0.79 0.72 0.67 0.66 0.65 0.68 0.64 0.71 0.74 0.61

0.54 0.57 0.55 0.63 0.40 0.18 0.26 0.28 0.64 0.67 0.62 0.59 0.25 0.35 0.45 0.38 0.40 0.35 0.48 0.44 0.15 0.17 0.26 0.29 0.34 0.33 0.46 0.42 0.12 0.27 0.24 0.17 0.21 0.23 0.26 0.20 0.27 0.65 0.41 0.39 0.47 0.50 0.29 0.43 0.26 0.22 0.33 0.43 0.26 0.25 0.19 0.17 0.25 0.45 0.51 0.31 0.21 0.21 0.19 0.20 0.29 0.33 0.28 0.38

t w Dify hain Flow oML wift eMo Lab flow Dify ChainGFlow ntoMLS-Swif NeMo acLab gC G ent S-S N Isaac Lang g e Isa Lan RA B M Lan RA B M

gflo

Lan

(b) Similarity of descriptive features.

(c) Jaccard similarity of modules.

Figure 5: Similarity assessment between repository semantics. The left panel compares generated semantics with human annotations, while the right panels report descriptive and module similarity between selected target repositories. TABLE 4: Comparison of performance and efficiency. Method

TP

FP

FN

TN

Acc.

Prec.

Recall

In/Out Token (M)

Claude Code Vulnhalla [39] I NFRA S COPE (Ours)

25 10 24

39 16 7

44 59 45

23 46 55

0.366 0.427 0.603

0.391 0.385 0.774

0.362 0.145 0.348

381.12/3.27 14.52/0.16 51.11/0.58

module names and different decomposition strategies. The high similarity pairs correspond to shared application contexts. LLaMA-Factory and ms-swift reach 0.87 descriptive similarity and 0.65 module Jaccard similarity because both implement training and model loading workflows. IsaacSim and IsaacLab also reach 0.87 descriptive similarity, reflecting their shared simulation context. On the Agent and RAG side, llama_index and dify have the highest module Jaccard similarity at 0.67, showing that both expose overlapping retrieval, integration, and workflow components. The low similarity pairs are also meaningful. llama_index and IsaacLab have only 0.60 descriptive similarity, and GPT-SoVITS and IsaacLab have 0.61, because their application contexts are different even though all belong to AI infra. Similarly, ms-agent and BentoML have a module Jaccard similarity of 0.12, indicating that an agent-tooling project and a serving framework share few module roles despite both exposing interfaces to models. These observations support the design of I NFRA S COPE, where the repository semantics preserve enough human-recognizable structure to guide localization, while the mutual similarity matrix distinguishes closely related repositories from superficially related ones.

5.3. RQ2: Comparison with Baselines After checking whether generated semantics are meaningful, we evaluate whether they improve the full workflow for vulnerability variant detection. Overall Detection. We evaluate I NFRA S COPE and baselines and gather the detected candidates. Then, we manually label each candidate as positive (i.e., potential vulnerability) and negative (i.e., non-exploitable). Finally, we deduplicate and unify the candidates and compare I NFRA S COPE with the baselines in Table 4, including candidates over the eight reference vulnerabilities. I NFRA S COPE reports 31 candidates, of which 24 are true positives and 7 are false positives,

TABLE 5: Transfer from reference to target among I NFRA SCOPE TP attributions. The benchmark contains 24 unique TPs; counts here are semantic attributions, and one candidate may be attributed to more than one reference CVE. Reference

Source pattern

Target repositories

CVE-2025-53002 CVE-2025-22603 CVE-2025-23348 CVE-2025-1793 CVE-2025-6984 CVE-2025-1040 CVE-2025-23361

Deserialization in adapters SSRF / URL validation bypass Training asset code injection Query injection XML / tool parsing abuse Template / code injection Command injection

BentoML, Megatron-Bridge, ms-swift autogen, langflow Megatron-Bridge, NeMo, Model-Optimizer langchain-community, ragflow autogen langflow Megatron-LM

# TP 12 6 5 2 2 3 2

yielding the highest precision (0.774) and accuracy (0.603) among the three tools. Its recall is lower than Claude Code because I NFRA S COPE performs inspection guided by the reference and dynamic validation before retaining candidates. This favors variants with high confidence over broad repository sweeps. Efficiency. I NFRA S COPE achieves the best tradeoff between effectiveness and efficiency in this benchmark. Claude Code is the most expensive baseline under the same workload based on reference CVEs, consuming 384.39M total tokens. I NFRA S COPE consumes 51.69M tokens, which is over 7× fewer than Claude Code, while detecting only 4% fewer true positives. Vulnhalla is cheaper because static scanning contributes no LLM tokens and its LLM reasoning step is smaller than agentic inspection and verification. This lower cost comes with lower recall (0.145) and precision (0.385) than I NFRA S COPE. I NFRA S COPE spends more tokens than the Vulnhalla pipeline filtered by SAST but improves both precision and recall, while using far fewer tokens than Claude Code and achieving higher accuracy.

5.4. RQ3: Detection Performance The aggregate metrics show that I NFRA S COPE favors precision over broad coverage. We next examine which reference mechanisms transfer successfully and which error categories explain the remaining false positives and false negatives. Transfer Across Repositories. Table 5 shows that the true positives found by I NFRA S COPE are not confined to the original applications. The strongest transfer signal comes from training and inference asset handling: the LlamaFac-

tory deserialization reference (CVE-2025-53002) leads I N FRA S COPE to unsafe model, runner, and checkpoint loading paths in BentoML, Megatron-Bridge, and ms-swift, while the NeMo and Megatron training references (CVE-202523361 and CVE-2025-23348) transfer to NeMo tokenizer and import utilities, Model-Optimizer conversion, TensorRT JSON serialization, and Megatron-Bridge dataset utilities, including three NeMo findings that have been assigned CVEs. This matches the expected module semantics of AI infra: vulnerabilities in post-training and model loading code often reappear in different repositories through adapter checkpoint or dataset handling rather than through identical APIs. False Positives. Manual examination shows that the 7 false positives mainly arise when dynamic validation confirms that a sink can execute, but the final deployment boundary is not well captured by the agent and does not support a reportable vulnerability. This occurs most often when the sink is an intended feature. For example, I NFRA S COPE treats an IsaacSim Jupyter executor that accepts code supplied by the user as suspicious, but the notebook is generally controlled by the authorized user and does not cross a new privilege boundary. Similar reasoning applies to the msagent local code executor, where code or shell execution is the advertised capability rather than an unexpected privilege crossing. The second cause involves trusted boundaries or boundaries controlled by the operator. For example, in ModelOptimizer, the suspicious checkpoint paths are search or training assets selected by the operator rather than assets shared by an attacker under the final policy. The third cause is source and sink overapproximation. For instance, LangChain Cassandra, Spark SQL, and mem0 LanceDB expose query or filter construction as interfaces for callers. In addition, some CLI paths also use safer execution forms than the source abstraction implies. These examples explain why dynamic verification reduces hallucinated reports but cannot eliminate every false positive: the remaining ambiguity is often in deployment boundaries rather than missing code evidence. In other words, the agent still needs stronger security policy knowledge to distinguish intended privileged functionality from an unintended attack surface. False Negatives. We also analyze the 45 missed candidates and summarize the causes and examples in Table 9 of Appendix C.3. The main reason is that the inspection is guided by the reference vulnerability semantics, whereas the baselines rely on a looser constraint (i.e., the vulnerability type). Furthermore, the inspection is less comprehensive than the baselines because we cap each repository at three iterations, and D EEP S EEK sometimes stops a tool-call turn early. GPT-SoVITS, which accounts for the most false negatives, illustrates this limitation: several unvisited files in the Web UI and audio processing modules pass user-controlled paths or text fields into shell commands. These misses show that a repository can contain many parallel UI entries that are not all covered by a reference-guided inspection budget. In this case, I NFRA S COPE did not start the relevant file inspection before reaching the maximum iteration limit (i.e.,

three iterations). When we increased the maximum number of iterations, the relevant candidates were reported after the eighth iteration, at roughly three times the cost. Another cause is that deserialization vulnerabilities can be missed when distributed asset loaders span modules that are not captured as relevant in the repository semantics. Overall, the false negatives are mainly coverage failures. They indicate that I NFRA S COPE is effective when the reference vulnerability points to a narrow functional context, and requires thorough references to improve coverage.

5.5. RQ4: Case Study of OpenClaw The project OpenClaw has attracted broad community attention. However, its architecture differs from the collected agent orchestration projects. Specifically, OpenClaw has an average semantic similarity of 0.48 to the agent projects in Table 3, lower than their mutual average of 0.53. It also contains parallel modules with the same functionality serving different user requirements. Therefore, it is a suitable case for testing the ability of I NFRA S COPE to find variants across modules in the same project, and we focus on variants within OpenClaw (i.e., S ⋆ = S ). We apply I NFRA S COPE to OpenClaw to examine whether one confirmed source vulnerability can guide additional target findings in the same project. We use the GHSA3hcm-ggvf-rch5 [40] as the reference vulnerability, which is an optional exec approval-allowlist bypass: the command substitution or backticks hidden inside double quotes are missed by string-level allowlist analysis but interpreted by the shell during execution. I NFRA S COPE uses this execution mismatch as the reference and identifies three similar cases. The first case, GHSA-****-****-8q4w, stays within the exec path: shell init-file options such as --rcfile can inherit allowlist trust from an approved script path while loading attacker-chosen initialization first. Similarly, I N FRA S COPE transfers the same check and use drift invariant to Discord voice ingress and finds GHSA-****-****-6hch, where channel, name, and stale-role validation gaps can bypass allowlist checks over channels and members. During examination of the inspection log, we found the agent drifts from the desired module and starts to search in irrelevant modules. This is because our module taxonomy is not designed for OpenClaw-alike repositories. After manually correcting this false module assignment, I NFRA S COPE further identified a third case GHSA-****-****-wchg, which allows Android Canvas WebView pages from untrusted origins to invoke the JavascriptInterface bridge and inject instructions into the app. Together, these findings signify that I NFRA S COPE uses vulnerability semantics to preserve the source invariant, while localization guides deep inspection from an adjacent shell-wrapper variant to a cross-module authorization flaw.

6. Related Work Security measurement and vulnerability evidence. Large-scale studies measure security patches, report repro-

ducibility, and vulnerability lifetimes in open source software [10], [41], [42]. They show that public vulnerability evidence and patch timelines have defensive consequences, but they do not ask whether disclosed mechanisms appear as variants across functionally related AI infra repositories. Our measurement instead characterizes functional context overlap and trigger-level similarity, then uses those findings to define an auditing task after disclosure. Recurring vulnerability. Recurring vulnerability detection (RVD) asks whether a disclosed vulnerability or an equivalent variant reappears in another codebase. Prior systems transfer code clones and reused components [7], [8], [9], vulnerable and patch signatures [8], [10], [11], taint and firmware exploitation signatures [12], [13], semantically equivalent statements [11], or black-box IoT interface signatures [14]. Cao et al. [29] further show that effectiveness depends strongly on recurrence characteristics and matching assumptions. I NFRA S COPE targets an orthogonal setting: the public reference case defines the semantics to preserve [43], [44] even after significant refactoring in another AI infra project as long as the underlying vulnerability features remain similar. LLM-assisted vulnerability detection. LLM-assisted detectors use language models as analyzers of a target codebase: they audit repositories [15], compare implementations with specifications [45], or strengthen static analysis with taint specifications [16], [46], [47], code property slices [25], constraints [26], and agent scaffolds [27]. Their primary objective is to find vulnerabilities in the inspected program, not to preserve the trigger semantics of a disclosed reference case across functionally related repositories. I NFRA S COPE augments reasoning with specialized tools and harnesses for variant detection, unlocking the full reasoning potential of LLMs. Vulnerability reasoning evaluations. A separate line of work evaluates whether generic LLMs and agents can reason about vulnerabilities at code or repository scale [17], [28], [48]. These studies characterize failure modes such as shallow interprocedural reasoning, incorrect guard interpretation, and high cost under project-scale context. Long context degradation further motivates constrained localization instead of prompting over an entire repository [18]. Their contribution is empirical measurement of model capability, whereas I NFRA S COPE turns a known vulnerability into a localized and verified cross-repository audit. AI software security and LLM-enabled remediation. LLMSmith and AgentFuzz study RCE and taint-style bugs in LLM-integrated applications and agents [49], [50]. MirrorFuzz transfers shared bug patterns across deep learning framework APIs [51]. AppAtch uses LLMs for vulnerability patching after localization, and Vul-RAG distills vulnerability knowledge from historical vulnerable and fixed examples [52], [53]. These works either analyze AI software attack surfaces or use LLMs for post-localization repair and knowledge reuse. They do not use a reference vulnerability to guide variant detection.

TABLE 6: Comparison with related work. •, ⊙, and ◦ denote direct, partial, and no direct support. Prior work Recurring vulnerability [9], [11], [12], [13], [14], [29] LLM-assisted detection [15], [16], [25], [26], [27], [45] LLM vulnerability reasoning [17], [28], [48] AI software vulnerability [49], [50], [51] LLM-based repair and knowledge reuse [52], [53] I NFRA S COPE

Uses reference semantics

Finds related code

Cross repo transfer

Checks target code

⊙

⊙

•

⊙

◦

◦

⊙

◦

⊙

◦

AI infra setting

◦

⊙

◦

⊙

◦

⊙

⊙

⊙

⊙

⊙

⊙

⊙

◦

⊙

◦

•

•

•

•

•

7. Conclusion and Discussion Through a measurement over 688 repositories and 251 related public vulnerabilities, we find that AI infra vulnerabilities often recur as variants within functionally similar modules. This observation turns public disclosures into postdisclosure audits: whether the same trigger semantics appear in related repositories under different implementations. We design I NFRA S COPE, a reference-driven auditing harness that combines semantic modeling, localized inspection, and target-side verification. Across our benchmark and case study, I NFRA S COPE uncovers over 20 zero-day vulnerabilities, including 11 acknowledged cases and 4 assigned CVEs. In our paper, we adopt D EEP S EEK as the primary backbone LLM for I NFRA S COPE. However, our implementation is model-agnostic and can be integrated with any LLM, as it is orthogonal to specific model capacities and relies only on the model’s reasoning and tool-use capabilities. As these capabilities, particularly context handling and reasoning, improve, the semantic modeling can become more accurate and the inspection process can be more comprehensive. The application of I NFRA S COPE is not limited to AI infra and can be extended to other domains characterized by rapidly evolving applications with multiple variants or counterparts. To enable such generalization, the module and vulnerability semantic modeling components need to become more autonomous. Currently, we rely on a domainspecific module taxonomy to extract semantics. While this approach is practically effective, it lacks generalizability when transferring to new domains without human-defined taxonomies. As future work, we plan to leverage sub-agents to automatically construct and maintain the taxonomy from available applications, provided that sufficient diversity in the input data is ensured.

References [1]

Y. Zheng, R. Zhang, J. Zhang, Y. Ye, and Z. Luo, “LlamaFactory: Unified efficient fine-tuning of 100+ language models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). Bangkok, Thailand: Association for Computational Linguistics, Aug. 2024, pp. 400–410. [Online]. Available: https://aclanthology.org/ 2024.acl-demos.38/

[2]

vLLM Project, “vLLM documentation,” Official documentation, 2026, accessed: 2026-04-27. [Online]. Available: https://docs.vllm.ai/

[3]

[4]

[5]

[6]

S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models,” in The Eleventh International Conference on Learning Representations, 2023. [Online]. Available: https: //openreview.net/forum?id=WE_vluYUL-X A. Chan, K. Wei, S. Huang, N. Rajkumar, E. Perrier, S. Lazar, G. K. Hadfield, and M. Anderljung, “Infrastructure for AI agents,” Trans. Mach. Learn. Res., vol. 2025, 2025. [Online]. Available: https://openreview.net/forum?id=Ckh17xN2R2 National Vulnerability Database, “Nvd - cve-2025-53002,” National Vulnerability Database, 2025, accessed: 2026-02-20. [Online]. Available: https://nvd.nist.gov/vuln/detail/CVE-2025-53002 Anthropic, “Claude mythos preview,” Anthropic Alignment Science technical blog, Apr. 2026, published: 2026-04-07. Accessed: 2026-04-08. [Online]. Available: https://red.anthropic.com/2026/ mythos-preview/

[7]

S. Kim, S. Woo, H. Lee, and H. Oh, “Vuddy: A scalable approach for vulnerable code clone discovery,” in 2017 IEEE Symposium on Security and Privacy (SP), 2017, pp. 595–614.

[8]

S. Woo, H. Hong, E. Choi, and H. Lee, “MOVERY: A precise approach for modified vulnerable code clone discovery from modified Open-Source software components,” in 31st USENIX Security Symposium (USENIX Security 22). Boston, MA: USENIX Association, Aug. 2022, pp. 3037–3053. [Online]. Available: https: //www.usenix.org/conference/usenixsecurity22/presentation/woo

[9]

S. Woo, E. Choi, H. Lee, and H. Oh, “V1SCAN: Discovering 1-day vulnerabilities in reused C/C++ open-source software components using code classification techniques,” in 32nd USENIX Security Symposium (USENIX Security 23). Anaheim, CA: USENIX Association, Aug. 2023, pp. 6541–6556. [Online]. Available: https: //www.usenix.org/conference/usenixsecurity23/presentation/woo

[10] F. Li and V. Paxson, “A large-scale empirical study of security patches,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 2201–2215. [Online]. Available: https://doi.org/10.1145/3133956. 3134072 [11] K. Huang, C. Lu, Y. Cao, B. Chen, and X. Peng, “Vmud: Detecting recurring vulnerabilities with multiple fixing functions via function selection and semantic equivalent statement matching,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 3958–3972. [Online]. Available: https://doi.org/10.1145/3658644.3690372 [12] S. Feng, Y. Wu, W. Xue, S. Pan, D. Zou, Y. Liu, and H. Jin, “FIRE: Combining Multi-Stage filtering with taint analysis for scalable recurring vulnerability detection,” in 33rd USENIX Security Symposium (USENIX Security 24). Philadelphia, PA: USENIX Association, Aug. 2024, pp. 1867–1884. [Online]. Available: https:// www.usenix.org/conference/usenixsecurity24/presentation/feng-siyue [13] H. Xiao, Y. Zhang, M. Shen, C. Lin, C. Zhang, S. Liu, and M. Yang, “Accurate and efficient recurring vulnerability detection for iot firmware,” ser. CCS ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 3317–3331. [Online]. Available: https://doi.org/10.1145/3658644.3670275 [14] H. Yang, J. Guo, S. Yang, G. Zhao, Q. Liu, C. Zhang, Z. Tan, L. Shan, Q. Zhou, M. Zhou, J. Tai, and X. Jia, “Iotbec: An accurate and efficient recurring vulnerability detection framework for black box iot devices,” in Proceedings of the Network and Distributed System Security Symposium (NDSS). San Diego, CA, USA: The Internet Society, 2026. [15] J. Guo, C. Wang, X. Xu, Z. Su, and X. Zhang, “RepoAudit: An autonomous LLM-agent for repository-level code auditing,” in Proceedings of the 42nd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, A. Singh, M. Fazel, D. Hsu, S. Lacoste-Julien, F. Berkenkamp, T. Maharaj,

K. Wagstaff, and J. Zhu, Eds., vol. 267. PMLR, 13–19 Jul 2025, pp. 21 083–21 100. [Online]. Available: https://proceedings.mlr.press/ v267/guo25n.html [16] Z. Li, S. Dutta, and M. Naik, “IRIS: LLM-assisted static analysis for detecting security vulnerabilities,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=9LdJDU7E91 [17] F. Li, J. Jiang, D. Chen, and Y. Xiong, “Llm-based vulnerability detection at project scale: An empirical study,” arXiv preprint arXiv:2601.19239, 2026. [18] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, “Lost in the middle: How language models use long contexts,” Transactions of the association for computational linguistics, vol. 12, pp. 157–173, 2024. [19] G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu, “Hybridflow: A flexible and efficient rlhf framework,” p. 1279–1297, 2025. [Online]. Available: https://doi.org/10.1145/3689031.3696075 [20] S. Xiao, Z. Liu, P. Zhang, and N. Muennighoff, “C-pack: Packaged resources to advance general chinese embedding,” 2023. [21] National Institute of Standards and Technology, “National vulnerability database,” Official vulnerability database, 2026, accessed: 2026-04-27. [Online]. Available: https://nvd.nist.gov/ [22] Open Source Vulnerabilities, “Open source vulnerabilities,” Official vulnerability database, 2026, accessed: 2026-04-27. [Online]. Available: https://osv.dev/ [23] GitHub, “Github advisory database,” Official vulnerability advisory database, 2026, accessed: 2026-04-27. [Online]. Available: https: //github.com/advisories [24] Huntr, “Huntr,” Vulnerability disclosure platform, 2026, accessed: 2026-04-27. [Online]. Available: https://huntr.com/ [25] A. Lekssays, H. Mouhcine, K. Tran, T. Yu, and I. Khalil, “{LLMxCPG}:{Context-Aware} vulnerability detection through code property {Graph-Guided} large language models,” in 34th USENIX Security Symposium (USENIX Security 25), 2025, pp. 489– 507. [26] X. Li, Y. Su, J. Liu, Z. Lin, Y. Hou, P. Gao, and Y. Zhang, “Vulsolver: Vulnerability detection via llm-driven constraint solving,” arXiv preprint arXiv:2509.00882, 2025. [27] Y. Nie, H. Li, C. Guo, R. Jiang, Z. Wang, B. Li, D. Song, and W. Guo, “Vulnllm-r: Specialized reasoning llm with agent scaffold for vulnerability detection,” arXiv preprint arXiv:2512.07533, 2025. [28] A. Yildiz, S. G. Teo, Y. Lou, Y. Feng, C. Wang, and D. M. Divakaran, “Benchmarking LLMs and LLM-based agents in practical vulnerability detection for code repositories,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vienna, Austria: Association for Computational Linguistics, Jul. 2025, pp. 30 848–30 865. [Online]. Available: https://aclanthology.org/2025.acl-long.1490/ [29] Y. Cao, S. Wu, R. Wang, B. Chen, Y. Huang, C. Lu, Z. Zhou, and X. Peng, “Recurring vulnerability detection: How far are we?” Proc. ACM Softw. Eng., vol. 2, no. ISSTA, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3728901 [30] C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez, “Memgpt: Towards llms as operating systems,” 2023. [31] N. Shinn, F. Cassano, A. Gopinath, K. R. Narasimhan, and S. Yao, “Reflexion: language agents with verbal reinforcement learning,” in Thirty-seventh Conference on Neural Information Processing Systems, vol. 36. New Orleans, LA, USA: Curran Associates, Inc., 2023, pp. 8634–8652. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2023/file/ 1b44b878bb782e6954cd888628510e90-Paper-Conference.pdf

[32] H. Jiang, Q. Wu, C.-Y. Lin, Y. Yang, and L. Qiu, “LLMLingua: Compressing prompts for accelerated inference of large language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 13 358–13 376. [Online]. Available: https://aclanthology.org/2023.emnlp-main.825/ [33] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” ACM Comput. Surv., vol. 55, no. 12, Mar. 2023. [Online]. Available: https://doi.org/10.1145/3571730 [34] GitHub, “Codeql documentation,” Official documentation, 2026, accessed: 2026-04-27. [Online]. Available: https://codeql.github.com/ docs/ [35] Agent Skills, “Agent skills,” Official documentation, 2026, accessed: 2026-04-22. [Online]. Available: https://agentskills.io/home [36] DeepSeek-AI, “Deepseek-v3.2: Pushing the frontier of open large language models,” 2025. [37] Anthropic, “Claude code documentation,” Official documentation, 2026, accessed: 2026-04-27. [Online]. Available: https://code.claude. com/docs/ [38] CyberArk, “Vulnhalla,” GitHub repository, 2025, accessed: 2026-0301. [Online]. Available: https://github.com/cyberark/Vulnhalla [39] S. Kosman, “Vulnhalla: Picking the true vulnerabilities from a codeql haystack,” CyberArk Threat Research Blog, Dec. 2025. [Online]. Available: https://www.cyberark.com/resources/threat-research-blog/ vulnhalla-picking-the-true-vulnerabilities-from-the-codeql-haystack [40] Open Source Vulnerabilities (OSV), “Osv: Ghsa-3hcmggvf-rch5 (openclaw),” OSV, 2026. [Online]. Available: https://osv.dev/vulnerability/GHSA-3hcm-ggvf-rch5 [41] D. Mu, A. Cuevas, L. Yang, H. Hu, X. Xing, B. Mao, and G. Wang, “Understanding the reproducibility of crowd-reported security vulnerabilities,” in 27th USENIX Security Symposium (USENIX Security 18). Baltimore, MD: USENIX Association, Aug. 2018, pp. 919–936. [Online]. Available: https://www.usenix. org/conference/usenixsecurity18/presentation/mu [42] N. Alexopoulos, M. Brack, J. P. Wagner, T. Grube, and M. Mühlhäuser, “How long do vulnerabilities live in the code? a Large-Scale empirical measurement study on FOSS vulnerability lifetimes,” in 31st USENIX Security Symposium (USENIX Security 22). Boston, MA: USENIX Association, Aug. 2022, pp. 359–376. [Online]. Available: https://www.usenix.org/ conference/usenixsecurity22/presentation/alexopoulos [43] H. Zhang, Y. Pei, J. Chen, and S. H. Tan, “Statfier: Automated testing of static analyzers via semantic-preserving program transformations,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2023. New York, NY, USA: Association for Computing Machinery, 2023, p. 237–249. [Online]. Available: https://doi.org/10.1145/3611643.3616272 [44] H. Wang, Z. Xu, H. Zhang, N. Tsantalis, and S. H. Tan, “Towards understanding refactoring engine bugs,” ACM Trans. Softw. Eng. Methodol., vol. 35, no. 5, Apr. 2026. [Online]. Available: https://doi.org/10.1145/3747289 [45] M. Zheng, C. Wang, X. Liu, J. Guo, S. Feng, and X. Zhang, “Rfcaudit: An llm agent for functional bug detection in network protocols,” arXiv preprint arXiv:2506.00714, 2025. [46] H. Zhang, Y. Pei, S. Liang, and S. H. Tan, “Understanding and detecting annotation-induced faults of static analyzers,” Proc. ACM Softw. Eng., vol. 1, no. FSE, Jul. 2024. [Online]. Available: https://doi.org/10.1145/3643759 [47] H. Zhang, Y. Pei, S. Liang, Z. Xing, and S. H. Tan, “Characterizing and detecting program representation faults of static analysis frameworks,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2024. New York, NY, USA: Association for Computing Machinery, 2024, p. 1772–1784. [Online]. Available: https://doi.org/10.1145/3650212.3680398

[48] S. Ullah, M. Han, S. Pujar, H. Pearce, A. Coskun, and G. Stringhini, “Llms cannot reliably identify and reason about security vulnerabilities (yet?): A comprehensive evaluation, framework, and benchmarks,” in 2024 IEEE Symposium on Security and Privacy (SP), 2024, pp. 862–880. [49] T. Liu, Z. Deng, G. Meng, Y. Li, and K. Chen, “Demystifying rce vulnerabilities in llm-integrated apps,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 1716–1730. [Online]. Available: https://doi.org/10.1145/3658644.3690338 [50] F. Liu, Y. Zhang, J. Luo, J. Dai, T. Chen, L. Yuan, Z. Yu, Y. Shi, K. Li, C. Zhou, H. Chen, and M. Yang, “Make agent defeat agent: Automatic detection of Taint-Style vulnerabilities in LLM-based agents,” in 34th USENIX Security Symposium (USENIX Security 25). Seattle, WA: USENIX Association, Aug. 2025, pp. 3767–3786. [Online]. Available: https://www.usenix.org/conference/ usenixsecurity25/presentation/liu-fengyu [51] S. Ou, Y. Li, L. Yu, C. Wei, T. Wen, Q. Chen, Y. Chen, H. Tang, and Z. Pan, “Mirrorfuzz: Leveraging llm and shared bugs for deep learning framework apis fuzzing,” IEEE Transactions on Software Engineering, 2025. [52] Y. Nong, H. Yang, L. Cheng, H. Hu, and H. Cai, “APPATCH: Automated adaptive prompting large language models for Real-World software vulnerability patching,” in 34th USENIX Security Symposium (USENIX Security 25). Seattle, WA: USENIX Association, Aug. 2025, pp. 4481–4500. [Online]. Available: https://www.usenix.org/ conference/usenixsecurity25/presentation/nong [53] X. Du, G. Zheng, K. Wang, Y. Zou, Y. Wang, W. Deng, J. Feng, M. Liu, B. Chen, X. Peng et al., “Vul-rag: Enhancing llm-based vulnerability detection via knowledge-level rag,” ACM Transactions on Software Engineering and Methodology, 2024. [54] National Vulnerability Database, “Nvd - cve-2025-1793,” National Vulnerability Database, 2025, accessed: 2026-02-20. [Online]. Available: https://nvd.nist.gov/vuln/detail/CVE-2025-1793 [55] ——, “Nvd - cve-2025-1750,” National Vulnerability Database, 2025, accessed: 2026-02-20. [Online]. Available: https://nvd.nist.gov/ vuln/detail/CVE-2025-1750

Appendix A. Ethical Considerations This study relies only on the public repositories and publicly disclosed vulnerabilities. All validation is performed on local checkouts and isolated sandbox environments controlled by the authors. We do not scan production services, interact with live deployments, test assets without authorization, or collect private user data. The study does not involve human subjects or private personal data, so institutional review board review is not applicable under this scope. The main beneficiaries are maintainers, downstream users, and defensive researchers who need to determine whether a disclosed vulnerability mechanism has variants in related AI infra repositories. The main risk is that information about similar mechanisms or validation paths could help attackers target projects whose vulnerabilities are not yet fixed. A secondary risk is that generated exploit code could be mishandled during analysis. We reduce these risks by limiting the study to repository code and public disclosures, running validation locally, and treating generated PoCs as restricted materials.

I NFRA S COPE reports a candidate only after verification against the target repository, so the LLM inference alone is never treated as a vulnerability claim. For findings that remain unfixed or are still under coordination, we withhold payloads, detailed traces, logs for individual cases, and identifiers that would make exploitation or target selection easier. We judge the remaining publication risk to be limited relative to its defensive value, because the paper reports aggregate methodology and results without exposing unresolved exploit details.

Appendix B. Measurement Protocol

Feature Checkpoint recovery FSDP or ZeRO Dependencies and environments Tests vLLM integration Quantization and export Container configurations Web interface

Stacks

Share

6 6 6 6 6 5 4 3

85.7% 85.7% 85.7% 85.7% 85.7% 71.4% 57.1% 42.9%

TABLE 7: Repeated training stack features. The table reports repeated code and workflow features across the same seven frameworks. Stacks (of 7) counts how many of the seven frameworks contain each feature. Cohort share is the same count divided by seven.

B.1. GitHub Search and Curation The broad repository search uses the GitHub Search REST API to query public GitHub repositories that are not archived and were created on or after January 1, 2023. Topic queries use topic seeds, and keyword queries search quoted seeds over repository names, descriptions, and README files. The seeds cover agent-orchestration projects, LLMOps, RAG, serving, inference, observability, retrieval pipelines, workflow runtimes, and tool calling. For this family, the topic seeds are “ai-agent”, “agent-framework”, and “agent-orchestration”. Other seeds include “llmops”, “rag”, “model-serving”, “inference-server”, “ai-infrastructure”, and “observability”. Retrieval pipelines, workflow runtimes, and tool calling use keyword seeds only. The keyword seeds are “rag” and “retrieval pipeline” for RAG, “model serving” and “inference engine” for serving and inference, “agent orchestration” and “workflow runtime” for agent orchestration, and “llm observability” and “tool calling” for observability. Before aggregation, we keep the GitHub information used in the paper and merge repeated results by repository. We then add direct AI infra repositories that appear in the public vulnerability disclosure dataset but were not returned by the 2023+ discovery queries, and record this provenance in the repository audit table. Two authors independently checked whether each retained repository provides reusable AI infra functionality and assigned a primary capability family, with a third author resolving disagreements. The exclusion rules remove tutorials and courseware, sample and template repositories, prompt collections, benchmark repositories, personal assistant or chat applications, generic UI or proxy tools without reusable AI infra functionality, model weight dumps, mirrors and translations, and showcases without substantive reusable infrastructure logic.

B.2. Training Cluster Measurement Protocol This section reports the repository collection and source tree listings used for the seven framework training cluster analysis in Figure 1c and table 7. The cohort contains LLaMA-Factory, ms-swift, axolotl, TRL, verl, OpenRLHF, and unsloth. For each project, we use the GitHub default branch as of April 2, 2026 and retrieve the recursive source

tree listing for that branch. The measurement covers source tree structure and functional indicators on the default branch. We encode each repository with 12 binary features spanning project structure and training workflow motifs. The structure side covers documentation and examples, tests, container and runtime configs, requirements files, pyproject.toml, setup files, and src/ layout. The motif side covers checkpoint and resume logic, Web UI interfaces, quantization and export paths, FSDP or ZeRO runtime support, and vLLM integration. Each feature is identified from repository paths using predefined indicators. Representative path indicators include tests, requirements files, Docker directories, checkpoints, gradio, and vllm. Accordingly, this is a measurement over repository paths rather than full semantic parsing. It should be read as a proxy for implementation features, not proof of identical behavior. From this binary matrix, Figure 1c reports lower triangle shared feature counts over the 12 binary features. Figure 1c also reports upper triangle README embedding similarities computed from the same seven framework cohort. Table 7 reports the motif prevalence table over the eight features used for the main text summary. In that table, Stacks (of 7) counts how many of the seven repositories contain the feature, and Cohort share reports the same count normalized by the seven repository cohort. Within this measurement range, counts of shared features are bounded by the same 12 binary features, while motif prevalence ranges from 0 to 7 repositories. In the current cohort, five of the eight measured features appear in 6 of 7 stacks and none appears in all seven.

Appendix C. Supplementary Results C.1. Variant within the Same Project Figure 6 shows variants within llama_index. CVE2025-1793 affects a Couchbase vector store integration, while CVE-2025-1750 affects a DuckDB vector store integration [54], [55]. The implementations differ in backend

CVE-2025-1793

CVE-2025-1750

105: class

60: class DuckDBVectorStore(BasePydanticVectorStore):

CouchbaseVectorStore(BasePydanticVectorStore):

314: def delete(self, ref_doc_id: str, **delete_kwargs: Any)

304:

-> None:

def delete(self, ref_doc_id: str, **kwargs:

Any) -> None:

322:

_ddb_query=f"""

314:

try:

323:

DELETE FROM {self.table_name}

315:

document_field=self._metadata_key

324:

WHERE 325: json_extract_string(metadata_,

+".ref_doc_id"

'$.ref_doc_id') = '{ref_doc_id}';

316:

self._scope.query(

325:

"""

317:

f"DELETE FROM

326:

ifself.database_name==":memory:":

`{self._collection_name}` WHERE {document_field} =

327:

self._conn.execute(_ddb_query)

'{ref_doc_id}'"

328:

else:

318:

329:

withDuckDBLocalContext(self._database_path) as_conn:

330:

_conn.execute(_ddb_query)

).execute()

319: logger.debug(f"Deleted document {ref_doc_id}")

Figure 6: SQL injection variants between two modules within llama_index. CVE-2025-1793 and CVE-2025-1750 (both are public records) occur in Couchbase and DuckDB vector store integrations but interpolate attacker-controlled document identifiers into SQL DELETE statements. API and file location, but both delete paths interpolate a document identifier controlled by an attacker into an SQL DELETE statement. This case complements the main text example across repositories: variants can appear across modules inside one project as well as across related AI infra projects.

Tool

Description

read_file

C.2. Inspection Agent Tools

get_function_code

Inspect code with line numbers and extract function/class bodies.

search_in_file

Agentic Tools. Table 8 shows the tools used by the inspection agent. The tool design follows the constrained interfaces exposed by the current coding agent [37] while keeping each action auditable.

search_in_folder

Locate sources, sinks, and semantic patterns via regex search.

list_files_in_folder Quickly summarize a module and its dependencies. get_imports analyze_data_flow

C.3. False Positive and False Negative

Summarize intraprocedural data propagation (parameters, assignments, calls, returns).

get_related_files

Table 9 summarizes the manual judgment categories behind the FP and FN analysis in RQ3, where FP rows summarize reported candidates rejected after human review, and FN rows summarize missed variants in the benchmark.

get_module_call_ relationships run_codeql_query read_codeql_results

C.4. Module Role Taxonomy

Execute C ODE QL queries and consume SARIF findings for verification.

read_shared public_memory

Table 10 shows the role taxonomy used for repository semantics extraction. We construct this taxonomy in three steps. First, three annotators label the top three reusable functionalities in 100 crawled repositories, producing 300 coarse raw module labels. Second, annotators group related labels and summarize disagreements within each group. Third, annotators decide whether each coarse category is necessary and keep at most five common second-level roles for retained categories. This procedure supports generalization across projects through coarse module categories while preserving domain-specific distinctions through second-level roles.

Expand analysis along static call relationships from the repository semantics.

report_vulnerability

Load prior observations as localization hints for the same target scope. Emit a structured candidate with location, supporting code facts, and confidence for later verification.

mark_file_completed check_file_status

Track progress in persistent analysis state and avoid duplicate inspection.

TABLE 8: Codebase analysis interface used by the inspection procedure.

TABLE 9: Summary of FP and FN categories. Examples are representative repository contexts used to explain the FP and FN analysis. Outcome

Category

Review cue

Representative examples

FP

Intended privileged functionality

Advertised capability for an authorized user

IsaacSim Jupyter executor ms-agent local executor

FP

Trusted deployment boundary Operator-controlled data or internal service control required

FP

Source and sink overapproximation

Caller-facing query or filter interfaces and safer CLI execution weaken attacker-control or sink assumptions

FN

Broad UI command paths

User controlled paths appear outside the localized budget

FN

Distributed data loaders

Checkpoint and adapter loaders span modules

FN

URL fetch and SSRF paths

Agent tools and workflow components expose many fetch paths

FN

Vector-store query construction

Query syntax is assembled in adapter-specific storage layers

BentoML remote runner response Model-Optimizer search checkpoint Megatron-Bridge checkpoint state mem0 LanceDB filters LLaMA-Factory SGLang argv launch GPT-SoVITS Web UI UVR audio processing paths GPT-SoVITS checkpoint loaders Transformers loaders Megatron-LM checkpointing AutoGPT request utility Dify HTTP request executor Langflow RSS and web search LangChain MyScale and SingleStoreDB LlamaIndex Couchbase

Module Categorization

Definition

Platform Systems

Build, package, configure, and orchestrate the runtime substrate for local, Build Packaging containerized, or distributed AI workloads. Runtime Hardware Distributed Orchestration Ingest, normalize, chunk, store, and retrieve datasets or external Ingestion Connectors knowledge used by models and applications. Dataset Construction Preprocess Tokenization Storage Formats Knowledge Stores Define model assets and loading paths, including architectures, Model Definition checkpoints, tokenizers, processors, and runtime configuration. Checkpoint Formats Loading Configuration Tokenizers/Processors Export Interchange Run training loops, distributed optimization, checkpointing, and Training Loop experiment configuration. Distributed Training Optimizer Schedules Checkpoint/Finetuning Experiment Configurations Adapt pretrained models through supervised finetuning, PEFT, preference Supervised Finetuning optimization, RLHF/RLAIF, or distillation style procedures. Parameter Efficient Finetuning Preference Learning RLHF/RLAIF Distillation/Quantization Aware Training Execute trained models efficiently through inference runtimes, cache and Inference Runtime memory control, parallelism, kernels, and performance measurement. KV Cache/Memory Inference Parallelism Quantized Kernels Performance Benchmarking Expose models or AI workflows through APIs, deployable services, Serving API routing, authentication, and runtime boundaries. Deployment Assets Autoscaling/Routing Authentication/Rate Limiting Multi Tenant Isolation Build retrieval pipelines that load documents, create embedding indexes, Document Loaders/Chunking retrieve and rerank context, and attach citations. Embedding/Indexing Retrieval/Reranking Citation Attribution Hybrid Search Implement agent control loops, tool or function calling, planning Tool/Function Calling orchestration, memory state, and plugin integrations. Planning/Orchestration Memory State Integrations/Plugins Measure quality, safety, performance, and regression behavior for models, Quality Evaluation pipelines, or applications. Safety Evaluation Performance Evaluation Regression Tests Track experiments, register models, collect traces, metrics, and logs, and Experiment Tracking support CI/CD or governance workflows. Model Registry Tracing/Metrics/Logs CI/CD Governance Provide web interfaces, workflow builders, CLI/developer workflows, Web UI templates, and examples. Workflow Builder CLI/Developer Workflows Templates/Examples

Data Knowledge

Model Assets and Loading

Training and Optimization

Post-Training and Alignment

Inference and Acceleration

Serving and Deployment

RAG and Retrieval

Agents and Tooling

Evaluation and Benchmarking

Observability and LLMOps

UI and Workflows

Second Level Module Role

TABLE 10: AI infra module role taxonomy Vrole used for repository semantics.