Do Skill Descriptions Tell the Truth? Detecting Undisclosed Security Behaviors in Code-Backed LLM Skills Wenhui He, Yue Li, Bang Fu, Huan Xing, Xing Fan, ZeHua Zhang, Baoning Niu
arXiv:2605.12875v1 [cs.CR] 13 May 2026
Skill Security Team Abstract—Programmatic skills in LLM ecosystems consist of a natural-language description and executable implementation files. Users and LLMs rely on the description to understand the skill’s scope. However, the implementation may perform security-relevant operations, such as credential access, network communication, or command execution, that the description does not state. We study this description–implementation inconsistency by asking whether the implementation stays within the securityrelevant scope declared in the description. We manually analyze 920 real-world programmatic skills and construct an 11-category security property taxonomy. Based on this taxonomy, we build S KILL S COPE, which constructs source-level security property graphs (SPGs) from implementations and performs LLM-assisted consistency checking. SPG nodes retain source-level code patterns rather than abstract taxonomy labels, preserving fine-grained evidence for checking. On 4,556 programmatic skills with doubleblind human review, S KILL S COPE achieves a precision of 84.8% and a recall of 96.5% for identifying inconsistency. Confirmed inconsistency affects 9.4% of skills, while cases of coarser description, in which implementation details remain within the declared scope, account for 24.3%. Ablation experiments confirm that both the SPG and the taxonomy contribute: removing the taxonomy reduces precision from 87.8% to 72.3%, while removing the SPG reduces recall from 94.7% to 79.0%. Index Terms—LLM skills, software security, specification– implementation consistency, program analysis, empirical study
I. I NTRODUCTION LLM skill ecosystems are expanding, and reusable skills have become a common way to package task-specific capabilities [1]– [3]. Some skills include executable code and a natural-language description file. Different platforms use different file structures, but they share a common pattern: a description file, often named SKILL.md, specifies the skill’s intended functionality in natural language. Some skills also include one or more implementation files that define executable behavior [1], [4]. In this work, we refer to skills with such implementation files as programmatic skills, distinguishing them from promptonly skills that rely only on natural-language instructions. Throughout this paper, we use description to refer to the entire SKILL.md file, including any declarative metadata header it provides (with fields such as description: and instructions:) and any free-form natural-language content. We use implementation to refer to the executable files shipped with the skill. Unless otherwise stated, “description” in this paper never denotes the description: metadata field alone.
Users and LLMs rely on the description to understand the skill’s scope before use, but the implementation can invoke external tools, access local files, communicate over the network, or handle credentials [4]–[6]. If the implementation performs security-relevant operations that the description does not state, users and LLMs may unknowingly invoke capabilities beyond what they intended. This raises a question: does the implementation stay within the security-relevant scope declared in the description? We define description–implementation inconsistency as cases where the implementation’s security-relevant behaviors exceed the scope declared in the description. Such inconsistency takes two forms. Undeclared behavior occurs when the implementation contains a security-relevant operation not covered by the description. Undeclared flow occurs when the implementation contains a data or control flow path among security-relevant operations that the description does not reflect. Not every description–implementation difference constitutes inconsistency. When the implementation operates within the capabilities already covered by the description but at a finer level of detail, we characterize the relationship as coarser description: implementation details remain within the declared scope but are more specific than what the description states. To study this problem, we manually analyze 920 real-world programmatic skills to derive a security property taxonomy of 11 first-level categories, producing a reference annotation set for description-side security properties. We then build S KILL S COPE, which constructs code-side security property graphs (SPGs) from skill implementations and performs LLM-assisted consistency checking using the taxonomy as a scope constraint. SPG nodes retain source-level operation patterns rather than abstract taxonomy labels, preserving finegrained evidence for checking. On the full dataset of 4,556 programmatic skills, S KILL S COPE achieves a precision of 84.8% and a recall of 96.5% for identifying description– implementation inconsistency. Ablation experiments on a 300skill subset confirm that both the SPG and the taxonomy contribute: removing the taxonomy reduces precision from 87.8% to 72.3%, while removing the SPG reduces recall from 94.7% to 79.0%. We make four contributions: • A security property taxonomy covering 11 categories of security-relevant behaviors, derived from manual analysis of 920 real-world programmatic skills, together with
a reference annotation set for description-side security properties. • S KILL S COPE , a tool that constructs source-level security property graphs from skill implementations and performs consistency checking against the description, achieving a precision of 84.8% and a recall of 96.5% on 4,556 programmatic skills. • An empirical analysis of 4,556 programmatic skills with double-blind human review of the full dataset, revealing that 9.4% exhibit confirmed inconsistency and 24.3% exhibit coarser description, and identifying the dominant inconsistency and granularity-mismatch patterns. • Ablation experiments and cross-model comparisons that quantify the contribution of each pipeline component. II. BACKGROUND AND M OTIVATION Figure 1 shows the structure of a programmatic skill. The description and implementation are located in the same skill directory but serve distinct roles: the description declares what the skill does, while the implementation defines how the skill executes [1], [7]. However, no mechanism enforces that the description covers all behaviors in the implementation, so the description may not fully reflect the security-relevant operations that the code actually performs. A. Motivating Example
programmatic skill/ +-- SKILL.md +-- scripts/ layer | +-- format_code.py implementation file
# Description layer # Implementation # Example
# SKILL.md --description: Format source code files using project conventions instructions: Read source files, apply formatting , write results --Apply the configured formatting style to each source file. # format_code.py def format_project(src_dir): files = discover_sources(src_dir) formatted = apply_style(files) write_formatted(formatted)
Fig. 1: Simplified structure of a programmatic skill. The SKILL.md file forms the description layer, while scripts or other implementation files define executable behavior. Our goal is to assess whether the implementation exceeds the security boundary declared in the description. We take the description as the reference boundary and check for undeclared behaviors or undeclared flows beyond that boundary. We do not attribute intent or determine whether a mismatch led to exploitation.
Consider a skill whose description states: “Read target files and run a fixed analysis workflow.” The implementation, however, contains os.getenv(‘API_KEY’) followed by requests.post(endpoint, headers={...}). Two III. S ECURITY P ROPERTY I NVESTIGATION forms of inconsistency are present. First, an undeclared behavDetecting inconsistency at scale requires a concrete definition ior: the implementation accesses a credential not mentioned in of what constitutes a security-relevant behavior and a dataset the description. Second, an undeclared flow: a local secret is of real-world skills to analyze. Figure 2 shows the overall read and transmitted to an external endpoint, forming a local- workflow. This section presents the construction of the dataset to-external data flow path absent from the description. Reading and the security property taxonomy. The remaining components, the description alone would not indicate either operation. SPG construction and consistency checking, are presented in In contrast, if a description states “read local files and Section IV. produce a summary report,” and the implementation writes the report to a specific path such as output/report.json, A. Dataset Construction this difference reflects finer detail within the scope already The dataset is based on a public snapshot collected as of covered by the description. We classify such cases as coarser March 5, 2026. We collect skill entries from four sources: two description rather than a security-boundary violation. first-party sources (Anthropic official skills and OpenAI) and two community aggregators (SkillsMP and skills.rest) [1], [3], B. Threat Model [8], [9]. Across these sources we obtain 22,515 raw skill items, We consider settings where users and LLMs derive their each linked to a public GitHub repository or directory. security expectations from the description before invocation, a) URL Normalization and Deduplication.: Links to while the implementation governs the actual runtime behavior. the same skill may take different forms across sources, A skill author, whether through oversight or intent, may and multiple links may also refer to the same skill diomit or understate security-relevant behaviors or flows in the rectory in the same repository. To unify these links, we description. parse each GitHub URL of the form https : / / github. com / We assume the following. The skill author controls the skill’s <owner > /<repo > /tree / <branch > /<path> into four comcontent, including both SKILL.md and the implementation ponents: owner, repo, branch, and subpath, where files. Users and LLMs treat the description as the security subpath is the repository-relative path after removing the boundary and do not independently audit the implementation repository and branch prefix. We then normalize subpath before use. The platform does not perform automated consis- to account for source-specific differences in directory laytency checking at the time of this study. out, for example mapping .claude/skills/<skill> to
.claude/skills, and retaining the nearest .../skills prefix for nested paths. After normalization, we use owner/repo@branch:normalized_subpath as the grouping key and merge links that map to the same download target. This deduplication reduces the 22,515 raw items to 11,049 unique download targets. b) Content-Based Filtering.: We then apply content-based filtering. Because the later analysis requires both a description and implementation files, we retain a skill directory only if it contains a SKILL.md file and at least one analyzable implementation file, identified by suffix (.py, .js, .ts, or .go). Skills that contain only SKILL.md, documentation files, images, data files, or other non-code files are excluded. Of the 11,049 unique download targets, 4,556 satisfy these criteria and form the final dataset of programmatic skills. Of these 4,556 skills, 17.6% come from the two first-party sources (Anthropic and OpenAI) and the remaining 82.4% from the two community aggregators. A single skill may include files in more than one of the four analyzable languages, so we do not report a strict per-language partition. B. Security Property Taxonomy
pilot subset itself is not limited to official samples. After the pilot, the annotators compare results, discuss disagreements, and align criteria and label boundaries. In the formal stage, both annotators independently annotate the remaining 800 skills under the aligned criteria. On the pilot subset, exactmatch agreement before adjudication is 90.0% (108/120) at the first level and 78.3% (94/120) at the second level; Table II summarizes dataset construction and the annotation setting. b) Conflict Resolution: For samples with disagreements or boundary uncertainty, the annotators first discuss the case based only on the original description text, the current taxonomy, and the annotation criteria; implementation files are not consulted at this stage. They focus on two questions: (i) does the statement explicitly state a description-side security property, and (ii) can the statement be covered by an existing property label. If the annotators still cannot reach agreement, the case is submitted to a designated adjudicator (the first author), whose decision determines the final label. c) Taxonomy Refinement: The taxonomy can be refined during annotation, but we do not update it based on a single new case. Such cases are first recorded for later review by the two annotators and the adjudicator. A label definition is revised, or a new label is added, only when the phenomenon cannot be covered by existing property labels and recurs across multiple samples with a stable boundary. After each update, the team applies the revised criteria consistently and, when necessary, revisits previously annotated samples. The taxonomy serves three roles in our pipeline: (1) it defines the label space for annotating description-side security properties; (2) it scopes code-side SPG node localization to the same set of categories; and (3) it provides a common reference for consistency checking by both S KILL S COPE and the human reviewers, so that their judgments are directly comparable.
We construct a two-level security property taxonomy that defines the scope of security-relevant behaviors considered in this work. The taxonomy contains 11 first-level categories and 32 second-level labels. The first-level categories cover securityrelevant behavior classes: file reads, file writes, command execution, network access, external API usage, secret access, dependency modification, system permission access, security control, observability, and infrastructure. The second-level labels specify the target of each behavior, such as API keys, session data, and identity information under SECRET_ACCESS. To reduce annotation ambiguity, the taxonomy also provides operational descriptions and boundary rules for overlapping IV. S KILL S COPE categories, such as NETWORK_ACCESS and EXTERNAL_API. Table I shows the full taxonomy. Building on the taxonomy and reference annotation set a) Annotation Workflow: We derive the taxonomy through from Section III, S KILL S COPE constructs a code-side security manual analysis of 920 programmatic skills, sampled from the property graph (SPG) from the implementation and checks 4,556-skill dataset in proportion to the source distribution; these whether the implementation exceeds the scope declared in the 920 annotated skills therefore form a subset of the evaluation description. set used in Section V. The annotation labels which descriptionFormally, for an implementation file f , the file-level SPG side security-property categories each SKILL.md explicitly is Gf = (Vf , Ef ), where Vf is the set of security property declares; it does not label inconsistency itself, which is judged nodes identified from security-relevant implementation sites only during the evaluation review. in f , and Ef ⊆ Vf × Vf is the set of directed intra-file edges We deliberately avoid LLM-assisted labeling at this stage: among these nodes, discovered through breadth-first search LLM-generated labels may infer capabilities that are not (BFS) reachability analysis on the intermediate representation. explicitly stated and may apply unstable boundaries to similar Each node v ∈ Vf corresponds to one security-relevant statements, merging them in some cases and splitting them in implementation location and is associated with its source others. Two authors therefore independently annotate the 920 location and source-level operation pattern. The graph contains skills, considering only security properties explicitly stated only security property nodes and excludes program nodes in SKILL.md without consulting implementation files or that are unrelated to the security properties considered in inferring unstated capabilities. The annotation proceeds in two this work. The BFS traversal follows only data-flow edges stages. In the pilot stage, both annotators independently anno- (from the program dependence graph, PDG) and control-flow tate the same 120-skill subset; we begin with official samples edges (from the control-flow graph, CFG); abstract syntax because their descriptions and implementation organization are tree (AST) structural edges are excluded to avoid spurious more regular, which helps stabilize the initial criteria, but the connections through syntactic containment. For a skill s with
Manual Benchmark Construction Skill Dataset Construction Official Skill Repositories
Third-Party Platforms
Anthropic
SkillsMP skills.rest
OpenAI
Manual Label Discovery SKILL.mdBased Manual Annotation (920 Skills)
Iterative Label Consolidation Two-Level Taxonomy First-Level Category
Two-Level Security Property Taxonomy
Description–Implementation Consistency Checking
Second-Level Label
FILE_READ
scripts
NETWORK_ACCESS
external services
....
....
SKILL.md Taxonomy
SPGs
Repository URL Canonicalization & De-duplication
Code-side Security Property Graph (SPG) Construction
.claude/skills/<skill> .claude/skills
Joern-based CPG Construction for Skill Code
Download with in-stage filtering Filtering URL
Retain only skills with code assets
Code skills
Skill implementation code
Joern
Security Property Node Extraction & File-level Graph Construction File-level Graph Merging & Inter-file Edge Completion
TaxonomyConstrained LLM Checking
Skill-level SPGs
Manual review of LLM checking
Inter-file edges from
programmatic skills (Total: 4,556)
imports, calls etc. Per-file SPG
Skill-level SPG
Fig. 2: Overview of the analysis pipeline. The pipeline includes dataset construction, taxonomy derivation and manual annotation from SKILL.md, code-side security property graph construction, and description–implementation consistency checking. implementation S files Fs , the skill-level S SPG is Gs = (Vs , Es ) with Vs = f ∈Fs Vf and Es = f ∈Fs Ef ∪ Escross , where Escross denotes cross-file edges connecting existing SPG nodes across files without introducing new nodes. A. Code-Side SPG Construction S KILL S COPE constructs an SPG from the implementation files of each skill following Algorithm 1. We first export a Code Property Graph (CPG) [10] of the skill folder using Joern [11] as the intermediate representation (Line 3). Joern’s CPG integrates AST, CFG, and PDG into a unified structure, providing both structural and flow-level information. SPG nodes are identified using taxonomy-derived localization rules, and edges are discovered through reachability analysis on the CPG. Joern is used only as the extraction source; after SPG extraction, the CPG is discarded. Because the localization rules are derived from the taxonomy rather than from the individual skill’s description, S KILL S COPE can detect security-relevant behaviors in the implementation even when they are not mentioned in the description. a) Node Localization: For each taxonomy category, we define a set of keyword-based localization rules that match security-relevant call sites in the source code. The rules operationalize the behavior classes defined by the taxonomy into source-level localization criteria: the taxonomy specifies what kinds of security-relevant behaviors should be recognized, while the localization rules specify what kinds of implementation sites are treated as evidence of such behaviors in code. For example, FILE_READ
is matched by patterns such as open( and readFile(; FILE_WRITE by f.write( and createWriteStream(; SECRET_ACCESS by os.getenv( and process.env.; and NETWORK_ACCESS or EXTERNAL_API by requests.post(, fetch(, and SDK-specific call patterns. Each matched site becomes an SPG node (Line 6); the rule set covers all 11 taxonomy categories, and the full pattern list is provided in our supplementary artifact. b) Node Representation: SPG nodes retain the source-level operation pattern at each matched site rather than abstracting it to a taxonomy label. For example, a node records os.getenv(‘API_KEY’) or requests.post(endpoint, headers={...}), preserving the specific call, argument, and target. We make this design choice because taxonomy labels are too coarse for accurate consistency checking: reading a private key file and reading a local JSON data file both map to FILE_READ, but their security implications differ substantially. Retaining source-level patterns allows the LLM to compare the concrete operation against the description rather than relying on an abstraction that omits security-relevant details. c) Edge Discovery: Given the set of SPG nodes Vf in a file f , we discover intra-file edges (Line 7) by checking, for each pair of nodes in Vf , whether a data-flow or controlflow path connects them in the CPG. We perform BFS on the CPG starting from one node; if another node in Vf is reached, we add a directed edge from the starting node to the reached node. The BFS traversal may pass through intermediate
TABLE I: Two-level description-side security property taxonomy. First-level label
Second-level label
Operational description
FILE_READ
FR-SCRIPT FR-REF FR-DATA FR-CONFIG
Read local scripts, source files, or implementation files. Read local reference documents, guides, templates, or instruction files. Read local input data, project files, datasets, or media files. Read local configuration, state, database, session, environment, or queue files.
FILE_WRITE
FW-OUTPUT FW-STATE FW-CONFIG FW-STRUCTURE
Write local output artifacts, such as reports or generated files. Write local state data, such as databases, caches, logs, metrics, queues, or other persistent state files. Write local configuration, environment, authentication, session, or browser-state files. Create directories, project structures, template instances, or scaffold files.
SYSTEM_COMMAND
SC-PY SC-CLI
Execute local Python scripts or skill-provided scripts. Execute shell, npm, npx, pip, docker, git, chmod, sqlite3, build, or test commands.
NETWORK_ACCESS
NA-WEB NA-SERVICE NA-DOWNLOAD
Access websites, web pages, browser content, web search, or localhost services. Access remote platforms, cloud services, model services, or SaaS systems. Clone, pull, or download remote repositories, images, models, datasets, or documents.
EXTERNAL_API
EA-DATA EA-PLATFORM EA-AI
Access remote databases, metadata services, or search APIs. Access remote platform APIs or business-system APIs. Access remote AI, model, inference, or search APIs.
SECRET_ACCESS
SA-KEY SA-SESSION SA-ID
Access API keys, passwords, OAuth tokens, access tokens, or similar secrets. Access cookies, browser sessions, authentication state, or similar session data. Access identity-related information, such as email addresses, mailto fields, or user identifiers.
DEPENDENCY_ MODIFICATION
DM-PKG DM-SYS DM-ENV
Install or update package dependencies through pip, uv, npm, or similar package managers. Install system-level dependencies, browsers, OCR tools, Playwright, TeX, or similar system components. Create or modify virtual environments, Conda environments, containers, or other execution environments.
SYSTEM_PERMISSION_ ACCESS
SPA-RESOURCE SPA-IAM SPA-VALIDATION SPA-ENFORCEMENT SPA-SECURITY
Access system resources or related permissions. Manage IAM roles or permissions. Perform security validation or permission checks. Enforce access restrictions or permission limits, including access-control and compliance checks. Perform security-related operations.
SECURITY_CONTROL
SEC-VALIDATION SEC-QUERY SEC-RATE
Validate inputs. Perform parameterized queries. Apply rate limiting.
OBSERVABILITY
OBS-LOG
Produce structured logs.
INFRASTRUCTURE
INF-HEALTH
Expose health-check endpoints.
TABLE II: Summary of dataset construction, manual annotation setting, and exact-match agreement. Item
Value
Public snapshot date Final dataset Manually annotated subset Pilot annotation (double-annotated) Formal annotation (double-annotated) Annotators Annotation source Annotation unit Adjudication Pilot first-level exact-match agreement Pilot second-level exact-match agreement
March 5, 2026 4,556 programmatic skills 920 skills 120 skills 800 skills 2 authors SKILL.md only One programmatic skill First author 108/120 (90.0%)
no direct CPG edge between the two security nodes, BFS discovers the path and adds an SPG edge. All reachable pairs in Vf × Vf contribute edges, forming the intra-file edge set Ef . d) Skill-Level Merging: After each file is processed, its SPG is accumulated into the skill-level Vs , Es (Lines 8–11). The algorithm then iterates over distinct file pairs and adds cross-file edges (Lines 12–14) based on explicit dependency evidence: import or require relations, resolved cross-file call references, and path-level dependencies. Crossfile edges connect existing SPG nodes across files without introducing new nodes; the resulting Gs = (Vs , Es ) is returned at Line 15.
94/120 (78.3%)
non-security program nodes that are not themselves part of Vf ; these intermediate nodes are not added to the SPG, but the reachability they establish produces an SPG edge. This approach captures indirect flows. For example, a credential read by os.getenv() may reach requests.post() through several intermediate variable assignments. Although there is
As an example of the resulting skill-level representation, the SPG for a PDF-processing skill with 8 implementation files contains |Vs | = 250 security-relevant nodes and |Es | = 301 directed edges. The nodes cover 18 distinct source-level operation patterns, including open(...), sys.argv[i], and json.load(...). This illustrates that the aggregation summarizes multi-file implementations through securityrelevant operations and their connections, rather than preserving the full program graph.
Algorithm 1: Code-side SPG construction for a programmatic skill Input : ps : folder path of a programmatic skill. R: property localization criteria derived from the taxonomy. C: cross-file edge completion criteria. Output : Gs = (Vs , Es ): skill-level code-side SPG. 1 Vs ← ∅, Es ← ∅ 2 S ← ∅ 3 CP Gs ← ExportCPG(ps ) // Joern exports CPG; used only as extraction source for SPG // Phase I: File-level SPG extraction
foreach implementation file f in ps do CP Gf ← GetFileSubgraph(CP Gs , f ) 6 Vf ← LocateSecurityNodes(CP Gf , R) // Taxonomy-derived
4
5
node localization 7
Ef ← DiscoverEdgesByBFS(CP Gf , Vf )
// BFS reachability
between SPG nodes 8 9 10 11
Gf ← (Vf , Ef ) S ← S ∪ {(f, Gf )} Vs ← Vs ∪ Vf Es ← Es ∪ Ef // Phase II: Skill-level merging and cross-file edge completion
foreach pair of distinct entries ((fi , Gfi ), (fj , Gfj )) in S do Ecross ← CompleteCrossFileEdges(Gfi , Gfj , C) 14 Es ← Es ∪ Ecross
12
13
15
return Gs = (Vs , Es )
B. Description–Implementation Consistency Checking
declared semantics, applying C1 to identify undeclared securityrelevant objects. Next, it examines each SPG edge, applying C2 to identify undeclared local-to-external flows. Finally, it produces two results: whether description–implementation inconsistency exists, and whether the skill exhibits coarser description. Throughout the process, the taxonomy constrains the LLM’s scope: it may only use labels defined in the taxonomy and must not infer capabilities unstated in the description. We set temperature=0 for deterministic output and make a single model call per skill. If code-side evidence is insufficient, the LLM returns an explicit uncertain status rather than a forced decision. c) Output Schema: For each skill, the LLM returns a structured JSON result rather than a single label. The output contains: (1) code-side evidence validation indicating whether the SPG evidence is sufficient for reliable checking; (2) declared semantics extracted from SKILL.md; (3) nodelevel results listing undeclared behavior candidates with their checking outcomes; (4) flow-level results listing undeclared flow candidates with their checking outcomes; (5) summary statistics covering the number of relevant nodes, flows, and flagged mismatches; (6) the final result on whether description– implementation inconsistency exists; (7) the coarser-description result; and (8) a cause summary for later manual review.
V. E VALUATION After constructing the SPG, S KILL S COPE checks whether the implementation exceeds the scope declared in the description. A. Experimental Setup We keep the description in its original text form and represent We evaluate S KILL S COPE on the full evaluation dataset only the implementation as an SPG, because the description of 4,556 programmatic skills, measuring how well its autois typically more abstract and forcing it into a graph would matic classifications agree with human judgment. The 4,556 introduce unstable node and edge alignment. The LLM takes SKILL.md files carry no pre-existing inconsistency labels; the three inputs: the description (SKILL.md), the serialized SPG ground truth used in this section is produced by the human (code_graph_json), and the security property taxonomy. review described next. a) Checking Criteria.: The checking produces two outTo produce that ground truth, two authors independently puts in a single pass. The first is whether description– review all 4,556 skills. The reviewers apply the same C1/C2 implementation inconsistency exists. We define inconsistency conditions and taxonomy as S KILL S COPE, ensuring a like-forbased on two conditions: like comparison. The two checkers use the same description• (C1) The implementation contains one or more security- side reference and taxonomy, but differ in their implementationrelevant objects that are not covered by any capability side input. S KILL S COPE takes SKILL.md, the SPG, and declared in the description. These objects include creden- the taxonomy as inputs, whereas each reviewer examines tial types, external entities, system permission scopes, or SKILL.md, the raw implementation files, and the same persistence targets that are neither mentioned nor implied taxonomy. The only difference is the representation of the by any declared capability. implementation evidence: S KILL S COPE sees a serialized SPG, • (C2) The implementation contains a data or control flow whereas reviewers read the raw source files directly. path that crosses security domain boundaries (e.g., local We call this protocol double-blind human review in two secret to external endpoint, user input to system command senses: each reviewer is blinded to the other reviewer’s execution), and the description does not cover this path. judgment, and both reviewers are blinded to S KILL S COPE’s If any node or edge in the SPG satisfies C1 or C2, the skill outputs. Disagreements are resolved through discussion, with is classified as exhibiting inconsistency. The second output unresolved cases referred to a designated adjudicator. The is whether the skill exhibits coarser description. This applies checking stage uses GPT-5 with a fixed prompt template and when the undeclared details remain within a capability already temperature=0, making a single model call per skill. declared in the description and do not satisfy C1 or C2. For ablation and baseline experiments, we use a random b) Prompt Design: Figure 3 shows the prompt template. sample of 300 skills from the 4,556-skill dataset. The size of The prompt structures the checking as a sequential process. The 300 balances statistical coverage with API cost: we evaluate LLM first extracts the security semantics explicitly declared in four configurations on this subset (the full system, two ablations, the description. It then examines each SPG node against the and one baseline), each invoking one LLM call per skill, so
Prompt template for description–implementation consistency checking System Role You are a security auditor for programmatic skills. Check whether the description in SKILL.md is consistent with the implementation represented by a code-side security property graph. Take SKILL.md as the reference boundary. Input Taxonomy: allowed security property labels and semantic scope. SKILL.md: the full description of the skill. code_graph_json: code-side security property nodes and edges. Audit Tasks Node consistency; flow consistency; overall consistency result. Core Principles Use only the taxonomy, SKILL.md, and code_graph_json. Do not infer unstated capabilities, flows, or labels. Return graph_extraction_uncertain if evidence is insufficient. Output Strict JSON containing declared semantics, node-level and flow-level results, and the two final checking results. Fig. 3: Prompt template for description–implementation consistency checking. It takes the taxonomy, SKILL.md, and the code-side SPG as input, and returns structured node-level, flow-level, and overall checking results. the total call count grows with both the subset size and the number of configurations. The same human-review protocol identifies 38 of these 300 as confirmed inconsistencies, which we treat as the ground truth on this subset; all configurations share these 300 skills and labels. For cross-model comparison, we use a smaller, deliberately constructed 30-skill subset that is stratified across our four collection sources and oversamples positive cases. The 9 inconsistencies among the 30 are confirmed by the same humanreview protocol applied to the full dataset, rather than preselected by any automated criterion. Two factors motivate this design. First, cross-model evaluation re-runs every skill under each candidate LLM, so the per-skill API cost is multiplied by the number of models compared. Second, under the natural inconsistency prevalence on the full dataset (about 9.4%), a random 30-skill sample would contain only 2–3 inconsistencies on average, too few to distinguish models on inconsistency recall. a) Metric: For description–implementation inconsistency, we use skill-level precision and recall: Precision = T P/(T P + F P ), Recall = T P/(T P + F N ), where T P is the number of skills flagged by S KILL S COPE and confirmed by manual review
as true inconsistency, F P is the number of flagged skills not confirmed as true inconsistency, and F N is the number of true inconsistency cases missed by S KILL S COPE. b) Compute and Human Effort: All local computation (CPG export via Joern, SPG node localization, and BFS edge discovery) was performed on a single machine running Ubuntu 22.04 LTS with an Intel Core Ultra 9 285K processor (24 cores), 46 GB RAM, and 1 TB storage. No GPU was used. On the full dataset of 4,556 skills, Joern CPG export took approximately 4 hours and SPG construction (node localization and BFS edge discovery) took approximately 2 hours. LLMbased consistency checking was performed via API calls to GPT-5 (main experiments), Gemini-2.5-flash, and Llama-3.370b (cross-model comparison); no local model inference was required. The GPT-5 checking stage for the full dataset (4,556 single-call invocations) took approximately 10 days, dominated by API rate limits and per-call latency rather than local computation. The human review of all 4,556 skills required approximately 9 working days per reviewer (two reviewers in parallel, each independently inspecting every skill). B. Detection Accuracy Table III shows the detection results on the full dataset. Among the 4,556 skills, S KILL S COPE flagged 487 as inconsistent. Of these, 413 were confirmed by human review (TP) and 74 were false positives. Human review also identified 15 false negatives, resulting in a precision of 84.8% and a recall of 96.5%. Table IV shows the full three-class confusion matrix. All 1,106 skills classified as coarser description were confirmed by human review, with none reclassified as inconsistency. The asymmetry follows from the two decision boundaries: coarser description is a containment question (whether an implementation detail falls within an already-declared capability), which admits a clear yes/no answer; inconsistency is a boundarycrossing question, where judging whether a declared capability is broad enough to cover a specific detail admits more variation and accounts for the 74 false positives. a) False-Positive Patterns: Table VII groups the 74 false positives into five patterns. The dominant error mode (FP1, 35 cases) is that S KILL S COPE treats finer-grained implementation detail as inconsistency even when the relevant higher-level behavior is already covered by the description; for example, a specific file path is flagged as an undeclared object even though the description states “read and process local files.” FP2 (23 cases) reflects situations where the description covers the platform/API/secret direction but does not itemize concrete flow-level or storage details. FP3 (12 cases) covers engineering, setup, observability, or execution choices treated as boundary expansion. FP4 and FP5 together account for 4 cases involving runtime evidence beyond SKILL.md or insufficient code-side evidence. b) False-Negative Patterns: The 15 false negatives fall into two patterns. FN1 (14 cases) stems from coverage limitations in the SPG construction stage: the keywordbased localization rules match direct call patterns such as
requests.post() and os.getenv(), but may miss equivalent operations expressed through SDK client objects, custom helper functions, framework-specific integration logic, or configuration construction whose call signatures do not match the predefined patterns. For example, an SDK call like client.chat.completions.create(...), which carries the configured API key to an external service, does not match the keyword rules and is therefore not localized as an SPG node. FN2 (1 case) reflects an input-presentation difference: API-access context can be more readily observed by reading the full source holistically than by traversing the more fragmented graph form, which we treat as a presentation issue rather than a graph-construction defect.
TABLE III: Inconsistency detection results on the full evaluation dataset (4,556 skills). Flagged
TP
FP
FN
Precision
Recall
F1
487
413
74
15
84.8%
96.5%
90.3%
TABLE IV: Confusion matrix on the full evaluation dataset. Human review System output Inconsistency (487) Coarser desc. (1,106) Consistent (2,963)
Inconsistency
Coarser desc.
Consistent
413 0 15
0 1,106 0
74 0 2,948
C. Component Contribution Table V shows ablation and baseline results on the 300-skill subset. S KILL S COPE (full) achieves an F1 of 91.1% on this are characterized as false negatives in Section V-B). Table VIII subset, consistent with the 90.3% on the full dataset (Table III). groups the 413 S KILL S COPE-captured cases into six patterns. Removing the taxonomy (w/o taxonomy) substantially in- The dominant pattern (IC1, 212 cases, 51.3%) is undeclared creases false positives. FP rises from 5 to 13, reducing precision credential usage and forwarding to external services: the from 87.8% to 72.3%. Without the taxonomy’s scope constraint, description may state a local task such as “analyze code the LLM treats finer implementation details as inconsistencies, style,” or describe the intended use of an external service, including cases that should instead be classified as coarser but does not cover that credentials will be read and then sent description. to that service. The implementation contains operations such Removing the SPG (w/o SPG), where the LLM receives the as reading an API key, constructing an authorization header, or raw source code instead, increases false negatives (FN rises attaching a token to an outgoing request. These cases combine from 2 to 8), reducing recall from 94.7% to 79.0%. Analysis undeclared behavior (credential access) with an undeclared of the missed cases shows that most involve cross-node flow flow (local data to an external service). This is followed by relationships. Raw source code contains substantial non-security undeclared flows from user input or local content to execution logic that can obscure security-relevant operations, and longer or system commands (IC3, 91 cases, 22.0%), and undeclared inputs may also be subject to context-window compression. local execution, permission changes, or dependency-related The SPG retains only security-relevant operations and their capability expansion (IC6, 54 cases, 13.1%). These results flows, providing a cleaner signal. The precision of w/o SPG show that confirmed inconsistency cases often involve both (90.9%) is slightly higher than S KILL S COPE (full), because the undeclared behaviors and undeclared flows rather than isolated cases that raw-code mode detects tend to be the most prominent node-level differences. inconsistencies. A further 1,106 skills (24.3%) were classified as coarser The baseline (code + SKILL.md, without either the SPG or description. Table IX shows their breakdown. The dominant the taxonomy) shows the lowest F1 of 73.0%, confirming that patterns are more specific local file handling (LU1, 419 cases) both components contribute to overall performance. and minor implementation-side details (LU2, 321 cases) such as Beyond the pipeline components, the choice of LLM backend logging, caching, or input validation that fall within an alreadycan also affect results. To quantify this, we re-run the full covered task scope. A representative coarser-description pattern S KILL S COPE pipeline on the 30-skill cross-model subset under is local state, cache, or configuration handling: the implementhree different LLMs. Table VI shows results with alternative LLMs. GPT-5 achieves the highest F1 (90.0%) and is used for tation includes state reads, cache access, configuration-file all main experiments, followed by Gemini-2.5-flash (85.7%) handling, or session-related operations that serve the declared and Llama-3.3-70b (72.7%). GPT-5 and Gemini-2.5-flash both capability without expanding the security boundary. These cases achieve 100% recall on this subset; their F1 gap is driven confirm that a substantial portion of description–implementation entirely by precision. Llama-3.3-70b has both lower precision differences reflect granularity mismatch rather than securityand lower recall, with its additional false positives concentrated boundary expansion. Taken together, the inconsistency and coarser-description in cases where finer implementation details are misclassified breakdowns point to a clear asymmetry: granularity-mismatch as boundary violations. cases concentrate on local-only details that stay within the D. Prevalence and Patterns declared task scope, whereas confirmed inconsistencies are Across the 4,556 skills, human review confirmed 428 cases dominated by undeclared flows that move local credentials, as inconsistent, giving a population prevalence of 9.4%. Of user input, or local content toward external destinations or local these, S KILL S COPE correctly captured 413 (the remaining 15 execution (IC1, IC3, IC6 jointly account for 357/413, 86.4%).
TABLE V: Ablation and baseline results on a 300-skill random subset (38 true inconsistency cases). Baseline feeds the LLM with raw source code and SKILL.md, without either the SPG or the taxonomy. Configuration
Flagged
TP
FP
FN
Prec.
Recall
F1
S KILL S COPE (full) w/o taxonomy w/o SPG
41 47 33
36 34 30
5 13 3
2 4 8
87.8% 72.3% 90.9%
94.7% 89.5% 79.0%
91.1% 80.0% 84.5%
Baseline
47
31
16
7
66.0%
81.6%
73.0%
TABLE VI: Cross-model comparison on a 30-skill subset with oversampled positive cases. Model GPT-5 Gemini-2.5-flash Llama-3.3-70b
Flagged
TP
FP
FN
Prec.
Recall
F1
11 12 13
9 9 8
2 3 5
0 0 1
81.8% 75.0% 61.5%
100% 100% 88.9%
90.0% 85.7% 72.7%
TABLE VII: Main false-positive patterns in the full evaluation dataset. ID
Pattern
Count
FP1 Declared capability already covers local I/O, persistence, or state/config handling. FP2 Declaration covers the platform/API/secret direction; concrete flow-level or storage details not itemized. FP3 Engineering, setup, observability, or execution details are implementation choices, not boundary expansion. FP4 Relevant declaration evidence exists at runtime beyond SKILL.md. FP5 Available evidence insufficient for a full inconsistency check.
35
Total
74
23
Pattern
1
IC3 Undeclared flows from user input or local content to execution or system commands. IC4 Undeclared external platform integration or data-destination changes. IC5 Code behavior directly exceeding the described security boundary. IC6 Undeclared local execution, permission changes, or dependency expansion. Total
Pattern
LU1 Code adds specific file paths, reports, or persistence locations under declared file handling.
LU2 Broad alignment; code contains only minor implementation-side details.
3
Count Examples
IC1 Undeclared credential usage and forwarding to external services. IC2 Undeclared local persistence, config reads/writes, or state writes.
ID
12
TABLE VIII: Confirmed inconsistency patterns in the full evaluation dataset. ID
TABLE IX: Main patterns for coarser-description cases in the full evaluation dataset.
212 docs-seeker, antigravity-quota 21 sequentialthinking, rag-query 91 clean-codereviewer, torch-compile 19 secret-scanner, theme-gen 16 playwright-skill, clipit 54 kotlin-in-action, create-designboard 413
VI. D ISCUSSION a) Implications: For skill developers, descriptions need not enumerate every implementation detail, but should clearly state the security-relevant capabilities, resources, and localto-external flows that define the skill’s security boundary. This distinction matters in practice: implementation details
LU3 Code adds logging, observability, or debugging details not itemized in description. LU4 Code adds cache, state, config, or session handling under a declared capability. LU5 Code introduces limited additional nodes (e.g., health checks, input validation). LU6 Code adds setup, directory creation, or packaging steps under a declared workflow. Total
Count Reason 419 Details serve the declared capability; no undeclared exfiltration or execution. 321 Differences are minor and do not support an inconsistency conclusion. 135 Observability details remain within the declared task scope. 108 Local state objects support the declared capability rather than extend it. 85 No dangerous flows; limited to small implementation-side details. 38 Workflow details rather than new capabilities. 1,106
may remain within an already declared capability without constituting a security-boundary violation. For platform providers, automated consistency checking can support review prioritization by surfacing skills with potential security-boundary violations. Distinguishing inconsistency from coarser description allows platforms to route high-priority inconsistency cases to human reviewers while treating coarserdescription cases as informational. Because S KILL S COPE requires only a single LLM call per skill after graph construction, the checking cost scales linearly with the number of skills, making it practical to integrate into skill submission or update pipelines. For end users, the checking results can serve as a transparency signal that surfaces the gap between what the description promises and what the implementation does, supporting
informed decisions before installing or invoking a skill. Host b) Security of LLM Agent Ecosystems.: Recent work LLMs that mediate skill invocation can consume the same examines security risks in LLM agent ecosystems, including signal at runtime to refuse or warn on flagged skills. empirical studies of skill-level risks [14], [15], attacks via b) Limitations: The SPG construction relies on static malicious skill files or hidden instructions [16], [17], and localization rules, which may miss security-relevant behaviors MCP-level threats such as tool poisoning [18], [19]. Defenses embedded in SDKs, dynamically loaded code, or uncon- include security auditing and runtime protection [20], [21]. ventional patterns. This pattern accounts for 14 of the 15 These studies focus on identifying risks or attack vectors. false negatives reported in Section V-B, and motivates either Our work addresses a different question: whether a skill’s extending the rule set or replacing keyword matching with a description is consistent with the security-relevant behaviors learning-based classifier over CPG nodes as future work. The in its implementation. current taxonomy covers the security properties observed in the c) LLMs for Program Analysis.: LLMs are increasingly analyzed dataset but may require extension as skill ecosystems applied to vulnerability detection and repair [22], [23], though evolve. The consistency checking depends on a single LLM prompting on raw code alone does not reliably outperform call per skill; multi-round or ensemble-based approaches may static analysis [24], [25]. Combining LLMs with static-analysis improve robustness but were not explored. outputs has shown improvements [26]–[28]. Our setting differs: c) Threats to Validity: The reported metrics depend on the we do not detect code vulnerabilities, but assess whether a model used (GPT-5) and may vary under other models, as the natural-language description is consistent with the security cross-model comparison in Table VI suggests. Human review behaviors in the implementation. This requires cross-modal involves inherent subjectivity, particularly at the boundary reasoning over text and structured code-side evidence. between inconsistency and coarser description; we mitigate d) Specification–Implementation Consistency.: Detecting this through independent double-blind review with discussion- inconsistencies between natural-language descriptions and code based conflict resolution and adjudication for unresolved has been studied in software engineering. Wen et al. [29] concases. Reported recall is an upper bound conditional on duct a large-scale empirical study of code-comment inconsistenSPG coverage: behaviors that the extraction rules do not cies. Panthaplackel et al. [30] propose deep-learning models for localize never reach the checking stage and are counted in just-in-time detection of comment-code inconsistency. Liu et neither TP nor FP. The dataset is drawn from four sources al. [31] apply code LLMs to detect and resolve code-comment (Anthropic official skills, OpenAI, SkillsMP, and skills.rest) mismatches. These works operate at the function or statement and may not represent all programmatic skill ecosystems. level, comparing inline comments or docstrings with adjacent Code-side SPG construction inherits the language coverage of code. Our setting differs in two respects: the specification is a Joern’s frontend (Python, JavaScript, TypeScript, and Go in standalone skill-level description (SKILL.md) rather than an our implementation); skills in other languages are excluded, inline comment, and the consistency question is specifically which may limit generalizability to ecosystems with different about security-relevant behaviors rather than general semantic language distributions. The analysis assumes non-adversarial alignment. skill authors; intentional obfuscation or evasion of consistency checking is not addressed. VIII. C ONCLUSION d) Reproducibility: We provide an anonymized impleWe study description–implementation inconsistency in promentation of S KILL S COPE at https : / / anonymous . 4open . science/r/SkillScope-4D66/. The repository contains the full grammatic skills by asking whether the security-relevant source code, the dataset collection and filtering scripts, the behaviors in the implementation remain within the descriptaxonomy and reference annotation set, the SPG construction tion’s declared scope. We construct a security property taxand consistency-checking pipeline, and evaluation scripts for onomy through manual analysis of 920 skills and propose reproducing the main experimental results reported in Section V. S KILL S COPE, which combines source-level security property The supplementary README includes environment configu- graph construction with LLM-assisted consistency checking. ration, data-access instructions, and step-by-step commands S KILL S COPE outputs both whether inconsistency exists and for reproducing detection accuracy, ablation, and cross-model whether the skill exhibits coarser description. Our evaluation on 4,556 skills with double-blind human comparison experiments. The repository will be de-anonymized review shows that S KILL S COPE achieves a precision of 84.8% upon acceptance. and a recall of 96.5%, with confirmed inconsistency affecting VII. R ELATED W ORK 9.4% of skills and coarser description accounting for 24.3%, a) Security Risks in Extensible Software Ecosystems.: confirming the necessity of distinguishing granularity mismatch Studies on package and extension ecosystems show that third- from security-boundary expansion. Ablation experiments and party components can introduce behaviors not visible from cross-model comparisons demonstrate the contribution of both their exposed interface [12], [13]. Programmatic skills share the SPG and the taxonomy. Future work includes extending this characteristic: users rely on the description, while the the taxonomy to cover emerging skill behaviors, exploring implementation determines runtime behavior. Our work focuses ensemble-based checking, and studying adversarial robustness specifically on the consistency between these two layers. against intentional evasion.
R EFERENCES [1] Anthropic, “Extend claude with skills,” https://docs.anthropic.com/en/ docs/claude-code/skills, 2026, accessed: 2026-04-14. [2] GitHub, “About agent skills,” https://docs.github.com/en/copilot/concepts/ agents/about-agent-skills, 2026, gitHub Docs. Accessed: 2026-04-14. [3] OpenAI, “Skills in chatgpt,” 2026, official documentation. [Online]. Available: https://help.openai.com/en/articles/20001066-skills-in-chatgpt [4] GitHub, “Creating agent skills for github copilot,” https://docs.github. com/en/copilot/how- tos/use- copilot- agents/cloud- agent/create- skills, 2026, gitHub Docs. Accessed: 2026-04-14. [5] Anthropic, “Claude code overview,” https://docs.anthropic.com/en/ docs/agents-and-tools/claude-code/overview, 2026, claude Code Docs. Accessed: 2026-04-14. [6] ——, “Security,” https://docs.anthropic.com/en/docs/claude-code/security, 2026, claude Code Docs. Accessed: 2026-04-14. [7] GitHub, “Adding agent skills for github copilot cli,” https://docs.github. com/en/copilot/how-tos/copilot-cli/customize-copilot/add-skills, 2026, gitHub Docs. Accessed: 2026-04-14. [8] SkillsMP, “Skillsmp: Agent skills marketplace,” https://skillsmp.com/, 2026, accessed: 2026-04-14. [9] skills.rest, “Agent skills library,” https : / / skills . rest/, 2026, accessed: 2026-04-14. [10] F. Yamaguchi, N. Golde, D. Arp, and K. Rieck, “Modeling and discovering vulnerabilities with code property graphs,” in 2014 IEEE symposium on security and privacy. IEEE, 2014, pp. 590–604. [11] Joern, “Overview | joern documentation,” https://docs.joern.io/, 2026, accessed: 2026-04-14. [12] M. Zimmermann, C.-A. Staicu, C. Tenny, and M. Pradel, “Small world with high risks: A study of security threats in the npm ecosystem,” in 28th USENIX Security symposium (USENIX security 19), 2019, pp. 995–1010. [13] R. Duan, O. Alrawi, R. P. Kasturi, R. Elder, B. Saltaformaggio, and W. Lee, “Towards measuring supply chain attacks on package managers for interpreted languages,” arXiv preprint arXiv:2002.01139, 2020. [14] Y. Liu, W. Wang, R. Feng, Y. Zhang, G. Xu, G. Deng, Y. Li, and L. Zhang, “Agent skills in the wild: An empirical study of security vulnerabilities at scale,” arXiv preprint arXiv:2601.10338, 2026. [15] Y. Liu, Z. Chen, Y. Zhang, G. Deng, Y. Li, J. Ning, Y. Zhang, and L. Y. Zhang, “Malicious agent skills in the wild: A large-scale security empirical study,” arXiv preprint arXiv:2602.06547, 2026. [16] D. Schmotz, L. Beurer-Kellner, S. Abdelnabi, and M. Andriushchenko, “Skill-inject: Measuring agent vulnerability to skill file attacks,” arXiv preprint arXiv:2602.20156, 2026. [17] Q. Wang, B. Ma, M. Xu, and Y. Zhang, “When skills lie: Hidden-comment injection in llm agents,” arXiv preprint arXiv:2602.10498, 2026. [18] X. Hou, Y. Zhao, S. Wang, and H. Wang, “Model context protocol (mcp): Landscape, security threats, and future research directions,” ACM Transactions on Software Engineering and Methodology, 2026, just Accepted. [Online]. Available: https://doi.org/10.1145/3796519 [19] Z. Wang, Y. Gao, Y. Wang, S. Liu, H. Sun, H. Cheng, G. Shi, H. Du, and X. Li, “Mcptox: A benchmark for tool poisoning on real-world mcp servers,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 42, 2026, pp. 35 811–35 819. [Online]. Available: https://doi.org/10.1609/aaai.v40i42.40895 [20] H. Zhang, Y. Nian, and Y. Zhao, “Agent audit: A security analysis system for llm agent applications,” arXiv preprint arXiv:2603.22853, 2026. [21] H. Hu, P. Chen, Y. Zhao, and Y. Chen, “Agentsentinel: An end-to-end and real-time security defense framework for computer-use agents,” in Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, 2025, pp. 3535–3549. [Online]. Available: https://doi.org/10.1145/3719027.3765064 [22] J. Wang, T. Ni, W.-B. Lee, and Q. Zhao, “A contemporary survey of large language model assisted program analysis,” Transactions on Artificial Intelligence, vol. 1, no. 1, pp. 105–129, 2025. [Online]. Available: https://doi.org/10.53941/tai.2025.100006 [23] X. Zhou, S. Cao, X. Sun, and D. Lo, “Large language model for vulnerability detection and repair: Literature review and the road ahead,” ACM Transactions on Software Engineering and Methodology, vol. 34, no. 5, pp. 145:1–145:31, 2025. [Online]. Available: https://doi.org/10.1145/3708522 [24] I. Ceka, F. Qiao, A. Dey, A. Valecha, G. Kaiser, and B. Ray, “Can llm prompting serve as a proxy for static analysis in vulnerability detection,” arXiv preprint arXiv:2412.12039, 2024.
[25] D. Gnieciak and T. Szandala, “Large language models versus static code analysis tools: A systematic benchmark for vulnerability detection,” IEEE Access, vol. 13, pp. 198 410–198 422, 2025. [26] Z. Li, S. Dutta, and M. Naik, “Iris: Llm-assisted static analysis for detecting security vulnerabilities,” in The Thirteenth International Conference on Learning Representations, 2025, iCLR 2025 Poster. [Online]. Available: https://openreview.net/forum?id=9LdJDU7E91 [27] X. Du, K. Yu, C. Wang, Y. Zou, W. Deng, Z. Ou, X. Peng, L. Zhang, and Y. Lou, “Minimizing false positives in static bug detection via llmenhanced path feasibility analysis,” arXiv preprint arXiv:2506.10322, 2025. [28] P. Li, S. Yao, J. S. Korich, C. Luo, J. Yu, Y. Cao, and J. Yang, “Automated static vulnerability detection via a holistic neuro-symbolic approach,” arXiv preprint arXiv:2504.16057, 2025. [29] F. Wen, C. Nagy, G. Bavota, and M. Lanza, “A large-scale empirical study on code-comment inconsistencies,” in Proceedings of the 27th International Conference on Program Comprehension (ICPC). IEEE, 2019, pp. 53–64. [30] S. Panthaplackel, J. J. Li, M. Gligoric, and R. J. Mooney, “Deep justin-time inconsistency detection between comments and source code,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 1, 2021, pp. 427–435. [31] H. Liu, Y. Wang, Z. Cai, P. Zhu, D. Zan, Y. Cui, B. Shi, and Y. Ma, “Docchecker: Bootstrapping code large language model for detecting and resolving code-comment inconsistencies,” in Proceedings of the 46th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2024, pp. 114–118.