How Agent Skills Evolve: A Large-Scale Empirical Study of Skills on GitHub

How Agent Skills Evolve: A Large-Scale Empirical Study of Skills on GitHub Jinjun Huang† , Jialun Cao‡ , Junxiao Han§ , Zhonghao Jiang† , Shing-Chi Cheung‡ , Zhongxin Liu†∗ † Zhejiang University, Hangzhou, China ‡ The Hong Kong University of Science and Technology, Hong Kong, China

arXiv:2611.04456v1 [cs.SE] 3 Nov 2026

§ Hangzhou City University, Hangzhou, China Emails: {huanghuanghuang, zhonghao.j, liu zx}@zju.edu.cn, {jialuncao, scc}@cse.ust.hk, hanjx@hzcu.edu.cn

Abstract—Large language model-based agents increasingly rely on external skills to reuse task-specific knowledge without retraining the underlying model. As skills become a common practice for extending agent capabilities, understanding creation and evolution is critical for building reliable agent systems. However, existing studies mainly examine agent skills from a static perspective, leaving their evolution in real-world repositories largely unexplored. This limits our understanding of the underlying patterns and latent motivations behind changes in skills. To fill this gap, this paper presents a large-scale empirical study of agent skills, capturing 84,200 skills and 750,898 related commits from GitHub. We manually construct taxonomies of skill components, change patterns, and evolution motivations, and build an automated annotation pipeline to label skill changes at scale. Based on the taxonomy, we conduct quantitative analyses of current skill composition and post-creation evolution frequency, together with component-level qualitative assessments of modified components, change patterns, and motivations. Our findings reveal five key implications: (i) reliability receives less attention in skill composition and change, motivating execution-contract skill design that incorporates verification and recovery mechanisms; (ii) evolution changes concentrate in project-coupled skills, calling for automated just-in-time skill updating mechanisms; (iii) change patterns are dominated by incremental enrichment and refinement, motivating pattern-guided skill enhancers that diagnose and improve incomplete skill components; (iv) redundancy is a practical maintenance issue, highlighting the need for contextefficient skill representation; (v) security-motivated changes show limited risk reduction, suggesting skill regression testing methods that objectively verify skill functionality and security. Beyond these insights, we envision our dataset to provide a basis for future studies and work on automated skill-related techniques.

the agent context as needed [12]. As illustrated in Figure 1, a skill is typically organized around a main instruction file (e.g., SKILL.md), which contains metadata for retrieval, workflow instructions, and task-specific knowledge, and may be accompanied by executable scripts and associated resources. By retrieving and composing skills during execution, agents can acquire domain-specific capabilities without retraining the underlying model, making skills an important building block for extending agent functionality [13], [14]. With models, tools, memory, and external knowledge (skills) orchestrated around agents, agent-based systems are evolving from prompt-response interfaces into a new form of software systems. In this form, foundation models serve as decisionmaking cores, tools provide external actions, memories preserve task context and experience, and skills encapsulate reusable execution knowledge. From a software engineering perspective, agent skills can be viewed as a new class of software artifacts. Skills are created, revised, reused, and maintained. They need to be continuously updated to changing task requirements, tool interfaces, execution environments, and user feedback. Studying these updates helps understand how skills evolve in practice and provides empirical guidance for writing, validating, and improving skills in the future.

The rapid evolution of the ecosystem surrounding agent skills has attracted increasing research attention [15], [16]. Empirical studies have been conducted to examine the acquisition, retrieval, and security risks of skills [17]–[19]. While these studies provide a static view of skills, the longitudinal evolution I. I NTRODUCTION of real-world skill artifacts is under-explored, i.e., how they Recent artificial intelligence (AI) systems have been shifting change across time is still unknown. This gap limits our from single-turn, passive, reactive large language models understanding of the underlying patterns and latent motivations (LLMs) to autonomous AI agents [1]–[4]. Unlike traditional behind changes in skills. To bridge the gap, we construct a LLM systems that primarily generate responses to user in- large-scale dataset of 84,200 agent skills and 750,898 related puts [5]–[7], AI agents are designed to accomplish task commits from real-world GitHub repositories and conduct an objectives autonomously: they can perform multi-step reasoning empirical study of skill evolution. By studying skills as evolving and planning, invoke external tools on demand, interact with software artifacts, we aim to learn how frequently they evolve, working environments, and maintain task context through which parts are modified, what kinds of changes developers memory mechanisms [8]–[11]. Among these capabilities, agent make, and what motivations drive these changes. skills are introduced as an important mechanism for packaging We study the following research questions and sub-questions. reusable task knowledge and execution procedures. A skill is a structured, reusable functional unit that can be loaded into • RQ1: What is the current status of agent skills in opensource repositories? We provide an overview of existing ∗ Corresponding author. skills, focusing on their composition and evolution frequency.

DIRECTORY

SKILL.md name: skill_name description: short skill summary workflow... metadata + workflow

>_ Code Scripts scripts/ ├── helper.sh ├── run.py └── .... executable scriptss

▯▯ Resources docs/ ├── reference.md ├── config.yaml └── .... supplementary contents

Fig. 1: File Structure of an Agent Skill – RQ1.1: What are the main components of agent skills? We manually construct a component taxonomy from inspected samples. Based on this taxonomy, we classify skill content into fine-grained functional components and quantify their occurrence in real-world repositories. The results show that most skills are currently used primarily to structure prompts, procedural guidance, and domain knowledge. At the fine-grained level, we observe that skills provide limited explicit quality assurance and failure-handling mechanisms. – RQ1.2: How frequently do agent skills evolve over time? We identify content-related modification commits as evolution events and analyze their counts and temporal intervals. Our findings show that most skills experience limited maintenance, and frequent evolution is concentrated in a small subset of complex and projectcoupled skills. This concentration suggests that projectcoupled skills incur high manual maintenance costs as their dependent projects evolve, highlighting the need for automated mechanisms that can efficiently propagate project-level changes into corresponding skill updates. • RQ2: Which components of agent skills are modified during evolution? Based on the collected dataset, we retrieve historical skill versions, parse file-level diffs, and use LLM-assisted component labeling to map changed content to functional components. We find that skill evolution is concentrated in instruction-oriented components. Within the instruction-oriented components, reliability-oriented components often receive less attention from developers. Given the inherent ambiguity of skills as natural language artifacts and the stochastic nature of LLM-based execution, the outcomes can be highly unpredictable. Reliability is therefore a nonnegligible concern for developers. • RQ3: What types of changes characterize agent skill evolution? To organize skill changes and their motivations under a unified framework, we sample evolution commits and manually construct a taxonomy of change patterns and latent motivations. Guided by the taxonomy, we aggregate file-level diffs into changed-component units, use LLMassisted labeling to assign each unit a how label, and quantify the resulting distribution of change patterns. The results indicate that change patterns is dominated by incremental enrichment and refinement, while structural reorganization is rare. This suggests that future automated skill maintenance tools may prioritize iterative enrichment- and refinementoriented support over structural-level revisions. Redundancy is also a practical maintenance issue in skill evolution, with developers frequently simplifying skill content by removing

unnecessary artifacts and duplicated instructions. This is especially beneficial for token saving in long-horizon agent task execution, where repeated skill invocations can otherwise lead to substantial context accumulation. • RQ4: What motivates the evolution of agent skills? Using the motivation dimension of the taxonomy constructed in RQ3, we also annotate each changed-component unit with a label and analyze the distribution of motivations. This analysis shows that skill evolution is dominated by Perfective maintenance, especially improving functionality and retrievability. In contrast, reliability- and risk-oriented maintenance receive limited attention. As AI agents frequently involve substantial risky operations, such as file modification and network access, ensuring the reliability and risk-prevention of skills that govern these operations should be treated as a maintenance priority. For security-related changes, we further compare the scanner-reported risks before and after the corresponding commits, and find that only a small fraction of changes are accompanied by observable reductions in risks. This gap between intended and observed risk reduction highlights the need for executable tests that can objectively verify skill functionality and security. To the best of our knowledge, we are the first largescale empirical study of agent skill evolution, providing a comprehensive understanding of the current status, evolution patterns, and motivations behind skill changes. In summary, this paper makes the following contributions: • Dataset. We construct and release a large-scale dataset capturing the evolution history of agent skills, comprising 84,200 skills and 750,898 commits from real-world GitHub repositories. This dataset provides a foundation for a wide range of empirical analysis. • In-depth Study. We conduct a comprehensive empirical study of agent skill evolution, uncovering characteristics, key patterns, and motivations behind skill changes. • Implications. Based on our findings in skill evolution, we discuss research opportunities in the skill ecosystem, focusing on automated skill refinement, skill testing, and contextefficient skill representation. II. BACKGROUND AND R ELATED W ORKS A. Agent skills: modular and reusable capabilities An agent skill is a predefined, reusable, task-oriented capability module that helps agents solve specific tasks. Unlike knowledge encoded in model parameters, skills are external artifacts that explicitly describe task-specific workflows, tool usage patterns, constraints, and domain expertise. This design allows agents to acquire new capabilities without retraining, reuse prior task experience across similar scenarios, and improve the consistency and reliability of task execution [20]– [22]. Structurally, a skill is typically organized as a standalone directory. As shown in Figure 1, it usually contains a core guidance file, such as SKILL.md, which includes retrieval metadata and execution workflow, and may include executable scripts or associated resources such as reference documents and configuration files.

Recent studies divide the lifecycle of agent skills into three stages: acquisition, retrieval/selection, and evolution [23]. A typical lifecycle starts with skill acquisition, where reusable procedural artifacts are created from human expertise or execution traces. These skills are then indexed and retrieved from skill libraries, selected according to the current task context. Based on execution feedback, existing skills can be modified and evolved over time. Following this lifecycle view, existing techniques can be grouped into skill acquisition, skill retrieval and selection, and skill evolution. Prior work addresses skill acquisition from interaction experience or external documents [24]–[26], skill retrieval and selection based on task context and constraints [27]–[29], and skill evolution through failure-driven revision [30]–[32]. Rather than proposing a specific technique, our work empirically studies real-world skill evolution from multiple perspectives.

identify active and potentially relevant repositories. Specifically, we retain repositories with more than three stars and more than three push events during the target period. We use these low thresholds as a conservative lower bound, ensure a minimal level of public visibility and recent development activity while retaining newly created repositories that may contain agent skills. This stage provides the initial pool of 196,970 repositories for subsequent skill file mining. Candidate Skill Collection. Based on the collected repositories, we identify files that potentially correspond to agent skills. Since skill files may follow different naming conventions and directory layouts across projects, we adopt a pattern-based search strategy. Specifically, we consider files matching common skill-related patterns, including skill.md, files under .codex/skills and .claude/skills, and Markdown files located in directories named skills. Following existing skill file specifications [35], we further apply heuristic rules to B. Empirical Studies on Agent Skills remove clearly invalid candidates. Specifically, we retain a skill As agent skills have rapidly emerged in agent ecosystems, file only when it provides the required metadata, including recent studies have begun to empirically characterize their non-empty name and description fields, and contains content, structure, and security [18], [19], [33]. Ling et al. [18] sufficient content, i.e., more than 100 bytes. This stage yields conduct a large-scale data-driven analysis of publicly listed 296,051 candidate skill files. Claude skills, examining their categories, adoption patterns, Evolution History Filtering. Finally, we filter the evolution length distribution, and redundancy. Liu et al. [19] further histories of the collected candidate skills. To ensure that study the security risks of agent skills at scale. They identify each skill has an analyzable evolutionary history, we require vulnerability patterns such as prompt injection, data exfiltration, each retained skill file to have more than three commits. privilege escalation, and supply-chain risks. Complementary to Here we define non-substantive commits as commits that are vulnerability analysis, another recent study [33] constructs a explicitly intended to fix minor textual errors. Concretely, we behaviorally verified dataset of malicious skills and reveals how identify potentially non-substantive commits based on commit malicious instructions, hidden behaviors, and attack payloads messages containing keywords such as typo, spelling, and appear in real-world skill ecosystems. grammar. Then we validate whether the remaining commits However, they mainly treat skills as static artifacts and focus contain content changes by checking their file-level patches, on characteristics of content, while the longitudinal evolution retaining only commits with non-whitespace added or removed of real-world skills is under-explored. Our work complements lines. After deduplication, we obtain a dataset composed of them by reconstructing skill evolution histories from commits 84,200 skills across 7,776 repositories, along with 750,898 and analyzing modified components, evolution patterns, and related commits. underlying motivations behind these changes. B. Analysis of Current Skill Status III. M ETHODOLOGY To address RQ1, we conduct a statistical analysis based on the dataset from two perspectives: structural composition and evolution frequency. Detailed Component Category. As discussed in II-A, the existing open format for agent skills mainly categorizes skill A. Dataset Construction content by file type [35]. However, file-level categorization To investigate how agent skills evolve in real-world settings, cannot capture the evolution of fine-grained functional contents a dataset that contains skills with evolutionary histories is within a skill. Therefore, we further classify skill content needed. This section describes our dataset construction pipeline, from a functional perspective to support fine-grained evolution including repository collection, candidate skill collection, and analysis. evolution history filtering. To determine these categories, 383 skill files are indepenRepository Collection. GitHub hosts over 300 million repos- dently and manually labeled by two authors of the study. itories. The first challenge lies in identifying high-quality Following prior work [36], [37], the sample size ensures a repositories that may contain skill files. As the concept of 95% confidence level with a ±5% margin of error. After the agent skills was formally introduced by Anthropic in October independent data annotation process, a collaborative review 2025 [12], we conservatively set September 1, 2025, as the was undertaken by the two authors to reconcile disagreements starting date of our collection window, which extends through and finalize the categories through consensus, thereby solidiApril 23, 2026. We leverage the GH Archive dataset [34] to fying the taxonomy used in our subsequent analysis. Finally, In this section, we present the methodology of our study. Our pipeline is described in Figure 2, from dataset collection to the analysis of collected data.

Dataset Construction (§3.1) Repository Collection GH archive events Github repositories

Candidate Skill Collection Pattern-based search

296,051

candidate repos

Non-empty name & desc

candidate skills

Content > 100 bytes

Current Skill Status(§3.2) • Component Taxonomy Metadata / Resources Workflow / Knowledge • Evolution Frequency

Evolution History Filtering

196,970

Diff validation

• Version-level Block Parsing • Component Labeling • Aggregation

84,200 Skills 750,898 Commits

Patterns & Motivations(§3.4)

Component Identification(§3.3) R Q 1

Remove non-substantive commits & #commit > 3

R Q 2

• Manual Taxonomy Construction • Automatic Annotation • Pattern & Motivation Analysis

R Q 3-4

Fig. 2: Overview of Methodology we established five top-level categories: metadata, workflow, knowledge, resources, and others, which are shown in Table I. Evolution Frequency Overview. To characterize the overall activeness of skill evolution, we reconstruct each skill’s postcreation evolution history from repository commits. We define a commit that modifies a specific skill as a skill evolution event, and exclude each skill’s initial creation commit, as our focus is on post-creation evolution rather than skill introduction. Based on these events, we measure: (1) the number and proportion of skills with at least one evolution event, (2) the number of evolution events per skill, and (3) the time intervals between consecutive evolution events. These metrics capture the prevalence, frequency, and temporal rhythm of skill evolution. C. Identification of Modified Skill Components To address RQ2, we further investigate where changes occur within skill content. Specifically, we take each skill evolution event as the unit of analysis and map the modified contents in each event to the predefined skill components based on the previously constructed component taxonomy. To illustrate this process, we provide an example in our online appendix1 . Step 1: Version-level Block Construction. A skill directory typically contains a main instruction file (i.e., SKILL.md) and several auxiliary resource files, such as scripts, data files, and supplementary documents. Since these two types of files play different roles in a skill, we process them separately. For the main instruction file, we retrieve both its pre-change and post-change versions for each evolution event. We parse each version into blocks based on Markdown format, including frontmatter, paragraphs, lists, tables, and code fences. Each block preserves its line range, block type, heading path, and textual content. This version-level block representation allows us to assign semantic components to the skill content in a more structured manner. For auxiliary resource files, we do not parse their internal content because these files are heterogeneous in format and lack a unified structure. Instead, we extract their file-level diffs from the commit patch as evidence for analyzing their concrete modifications. Step 2: Component Labeling and Change Localization. After constructing Markdown blocks for both versions of 1 https://github.com/ZJU-CTAG/SkillEMP

SKILL.md, we map these blocks to our predefined skill component taxonomy. Specifically, we provide the full content as context and ask an LLM to assign each target block to exactly one skill component. The main instruction file is classified into components, including metadata, workflow, and knowledge. Auxiliary resource files are also categorized by file type, including knowledge and resources. For labeling, we use DeepSeek-V4-Flash [38], as it provides strong semantic understanding capability at a reasonable cost for large-scale labeling. We evaluate the reliability of automated annotation in Section V-A, where the component-labeling task achieves an accuracy score of 0.881 against manually labeled data. We then perform line-level analysis for the commit diff between the pre-change and post-change versions. For modifications, the removed side is mapped to blocks in the pre-change version, and the added side is mapped to blocks in the postchange version. Pure deletions and insertions are mapped to the pre-change and post-change versions, respectively. Each changed line is mapped to its corresponding labeled Markdown block, thereby identifying the affected skill component. For auxiliary resource files, we map the file-level change to the component label assigned to the corresponding file. The detailed prompt for labeling is provided in our online appendix1 . Step 3: Aggregation. Finally, we aggregate the labeled changes at the granularity of a skill evolution event. If multiple changed Markdown blocks or auxiliary files in the same evolution event are mapped to the same component, we merge them into one changed component. As a result, each evolution event is represented as a set of changed skill components. Based on this representation, we compute the frequency and proportion of modifications for each skill component. This enables us to identify which parts of agent skills are more likely to evolve and provides the basis for further analyzing their evolution patterns and motivations. D. Taxonomy of Evolution Patterns and Motivations To address RQ3 and RQ4, we construct a two-dimensional taxonomy of skill evolution on top of the previously annotated skill components. Specifically, the “how” dimension captures the change operation applied to a modified component, while the “why” dimension captures the evolution motivation behind the modification.

TABLE I: Taxonomy of skill components. Component

Subcomponent

Definition

Metadata

Identity Functionality Trigger condition

Basic information that identifies a skill, such as name, version, author, or category. Descriptions of the skill’s capability. Conditions that determine when the skill should be retrieved or invoked.

Workflow

Execution Verification Recovery Format specification

Step-by-step procedures and operational logic to obtain results Steps for validating intermediate or final results during the workflow. Error-handling or fallback strategies for failed executions. Specifications of inputs, outputs, and intermediate results

Knowledge

Internal guidance Complementary documents

Background knowledge or domain-specific guidance embedded in Skill.md. External reference documents that supplement the main skill file.

Resources

Code scripts Data files

Executable scripts or program files used during task execution. Auxiliary data files, such as configurations, credentials, or dependencies.

Others

Miscellaneous

Other files that do not fall into the above categories, like evaluation files.

Fig. 3: Taxonomy of skill evolution motivations and patterns. Colored nodes denote fine-grained “why” categories, while white nodes denote concrete “how”. Taxonomy Construction. We randomly sampled 384 commits how label and one why label based on the commit message, to annotate, following the same independent annotation and localized diff, and surrounding skill context. consensus review procedure in III-B. This sample size also Based on these manually inspected samples, we first annotate satisfies the minimum requirement for a 95% confidence level the modified components in each skill evolution event following with a ±5% margin of error. To further assess the stability of the same procedure described in Section III-B. After obtaining the taxonomy, we additionally inspected 20 randomly selected the component labels, we map the changed lines in each commit commits after the initial taxonomy was constructed. In total, to their corresponding components. Then, for each component, our manual inspection covered 404 commits across 27 skills. we categorize the corresponding change operations and infer the The additional inspection did not lead to any changes to the maintenance intents based on the commit messages, changed taxonomy, indicating that the schema was stable for our dataset. lines, and skill content. We report the annotation reliability For each sampled evolution commit, we first inspect the for both modified component identification and evolution pre-change and post-change skill contents together with file- pattern/motivation labeling in Section V-A. The accuracy scores level diffs to identify the modified components. After the for component, change-pattern, and motivation labels are 0.913, modified components are determined, annotators use each 0.816, and 0.840, indicating that the automated annotations changed component as the annotation unit and assign one are highly consistent with human labels. During taxonomy

construction, we drew on the classical software maintenance taxonomy [39], [40] and adapted it to the agent skill. Taxonomy Structure. Figure 3 presents the hierarchical structure of our taxonomy for skill evolution. The taxonomy is organized into four top-level maintenance intents: Corrective, Adaptive, Perfective, and Preventive Maintenance. Each motivation is further decomposed into fine-grained why categories, shown as colored nodes, which are associated with concrete how operations, shown as white nodes. In this way, the taxonomy connects the reason for a skill change with the specific operation through which the change is implemented. Corrective Maintenance refers to changes that resolve deviations between a skill’s current behavior and its intended design goal. Such changes address missing, incorrect, or unstable skill content, including incomplete functionality, erroneous workflow steps, inaccurate knowledge descriptions, and underspecified execution conditions. Adaptive Maintenance captures changes motivated by external shifts rather than defects in the skill itself. These changes are needed when the surrounding toolchain, execution environment, interface, data format, or target task evolves, requiring the skill to be realigned with its updated usage context. Perfective Maintenance describes changes that improve a skill that is already functional. These changes go beyond basic correctness by enhancing skill capability, efficiency, and usability, reflecting developers’ efforts to make skills more effective in supporting future task execution. Preventive Maintenance denotes changes that proactively reduce future risks or maintenance costs. Although the current skill may remain executable, it may expose potential security, robustness, traceability, artifact-management, or evaluation risks. Preventive changes aim to mitigate such risks before they lead to failures. A detailed explanation of the fine-grained categories is provided in our online appendix1 . Automatic Annotation. Based on this taxonomy, we implement an automatic annotation pipeline. Consistent with Section III-C, we also use DeepSeek-V4-Flash as the annotation model. To avoid the coarse granularity of commit-level annotation and the excessive fragmentation of line-level annotation, we use the changed component defined in III-C as the basic annotation unit. Specifically, each changed component is represented as the tuple ⟨commit, file scope, component⟩. For changed components in SKILL.md, we provide the model with commit metadata, the target component, and block-level diffs with the full file as context. The block-level diffs include component type, heading path, and localized diffs. For changed components in auxiliary files, we extract the corresponding per-file diff from the original commit patch and provide it together with the file path and component label. Based on the taxonomy and the above artifacts, the model is required to produce a strictly structured JSON output containing one how label, one why label, and the rationale for the selected labels. In detail, the model needs to select labels from the predefined taxonomy and return exactly one how and one why category. When the predefined taxonomy is insufficient

Fig. 4: Distribution of Major Skill Components

Fig. 5: Distribution of Fine-Grained Skill Components to explain the change, the model is allowed to suggest a new category as an extension. The detailed prompt for annotation is provided in our online appendix1 . IV. R ESULTS A. RQ1: The Current Status of Skill Evolution 1) RQ1.1: Main Components Distribution: The first research question aims to identify the main functional components that constitute agent skills. To answer this question, we apply the component taxonomy defined in Table I to the collected skills and quantify the occurrence of each component. At the top level, Figure 4 shows that the three core components, i.e., metadata, workflow, and knowledge, appear in 99.9%, 98.2%, and 97.8% of skills, respectively. In contrast, resources are present in only 50.6% of skills, with code scripts appearing in only 14.6% of skills. These results suggest that skills are currently used primarily to structure prompts, procedural guidance, and domain knowledge. The relatively lower prevalence of code scripts further indicates that many skills do not package custom executable logic, and may instead rely more heavily on natural-language instructions, the agent’s general reasoning capabilities, and tools available in the execution environment. At the fine-grained level, Figure 5 shows that 92.5% of skills contain functionality descriptions, while only 35.2% of skills include verification steps, and only 49.6% include recovery instructions for handling failures. This indicates that existing skills often lack support for reliable execution. A possible

(a) Major Component Distribution

Fig. 6: Frequency and Temporal Rhythm of Skill Evolution explanation is that developers primarily focus on the task specification and execution guidance, implicitly relying on the agent’s own execution capability and human intervention to handle failures, leaving verification and recovery overlooked in skill creation. Moreover, 28.9% of skills lack trigger conditions and 7.5% lack functionality descriptions, which may make it harder for agents to retrieve and select. Findings 1: Skills are currently used primarily to structure prompts, procedural guidance, and domain knowledge. At the fine-grained level, they provide limited explicit quality assurance and failure-handling mechanisms. 36.4% of skills are incomplete in the metadata component, potentially hindering accurate retrieval by agents.

(b) Fine-grained Component Distribution

Fig. 7: Distribution of Changed Skill Components subset of complex and project-coupled skills, calling for future developers to focus on them and explore automated maintenance techniques that reduce developers’ maintenance cost. Skill evolution is also temporally concentrated, with most changes occurring within short-term windows. B. RQ2: Modified Components in Skill Evolution

To analyze where skill modifications occur, we apply component-level change analysis described in III-C to actively 2) RQ1.2: Evolution Overview: We reconstruct the evolution evolved skills. Because applying this analysis to all skills is history of each skill from repository commit histories. Among computationally expensive, we retain skills from repositories the collected skills, 64.3% have at least two valid commits. with more than 40 stars and skill files with more than 10 After applying the filtering criteria described in Section III-A, commits. The star threshold filters low-visibility repositories, there are 84,200 skills with analyzable evolution histories for while the commit-count threshold ensures sufficient evolution information for stable component-level analysis. This step the following frequency and temporal analyses. As shown in Figure 6(a), 58.5% of skills undergo fewer than results in 5,628 skills for the following analysis. Figure 7 presents the distribution of changed major skill five evolution events and 4% of skills undergo more than 20 evolution events, which suggests that most skills experience components. Workflow, knowledge, and metadata together limited post-creation maintenance. We further inspect these account for 86.0% of all component-level changes. Among frequently evolved skills and find that 86.9% of them are these components, workflow changes are the most frequent, packages that include auxiliary documents, scripts, or project- accounting for 40.0% of all component-level changes, while specific resources. This pattern reveals that complex and project- resources and others together account for only 14.1%. One coupled skills are more prone to be revised. As projects possible explanation is that resources and others are much frequently update, skills also need to be updated, which imposes less prevalent in skills, appearing in only about half as many a high cost for manual skill maintenance. Figure 6(b) shows skills as workflow (see Figure 4). Another reason may be that 83.1% of consecutive evolution events occur within one that effective workflows usually require iterative refinement. week, while only 6.0% exceed one month. This indicates that This is further supported by the RQ4 result that Workflow developers often make rapid adjustments based on a skill’s Improvement accounts for 18.05% of all labeled changes. early usage feedback until the skill becomes relatively stable. Findings 3: Among actively evolved skills, skill evolution is concentrated in instruction-oriented components. Findings 2: 58.5% of skills experience limited maintenance, and frequent evolution is concentrated in a small At the fine-grained level, workflow execution (18.6%) and knowledge internal guidance (14.7%) are the most frequently

TABLE II: Distribution of Change Patterns. How category

Share

Knowledge Enrichment Workflow Refinement Traceable Metadata Change Knowledge Modification Workflow Extension Knowledge Alignment Format Alignment Description Refinement Workflow Modification Metadata Refinement Interface Alignment Unnecessary Artifact Cleanup Expression Simplification Duplication Reduction Resource Modification Workflow Simplification Workflow Alignment Content Formalization Knowledge Restructuring Resource Alignment Others

9.00% 8.50% 7.59% 7.34% 6.14% 5.37% 5.01% 4.93% 4.47% 4.46% 4.15% 3.99% 3.63% 3.44% 2.09% 2.07% 1.97% 1.89% 1.74% 1.44% 10.79%

modified subcomponents. The reasons may be that they directly shape agent execution. In contrast, verification (4.7%) and recovery (7.1%) are modified less frequently, suggesting that developers pay comparatively less attention to result-checking and failure-handling logic. Given the inherent ambiguity of skills as natural language artifacts and the stochastic nature of LLM-based execution [41], [42], the outcomes can be highly unpredictable. The reliability of skills is therefore a non-negligible concern for developers. Findings 4: Fine-grained skill evolution mainly targets execution-oriented content, while reliability-oriented components often receive less attention from developers. C. RQ3: Changed Patterns in Skill Evolution

Fig. 8: Distribution of Motivations w It is also noted that simplification is a recurring maintenance behavior in skill evolution. Unnecessary Artifact Cleanup (3.99%), Expression Simplification (3.63%), Duplication Reduction (3.44%), and Workflow Simplification (2.07%) together account for 13.13% of all labeled changes. For example, 312362115/skills/deep-research removed the automatic update-check workflow that repeatedly ran after task completion to reduce redundant execution steps and unnecessary tool overhead. These changes suggest that developers sometimes encounter context-length pressure issues and attempt to simplify skill content by removing unnecessary artifacts, reducing duplicated instructions, and streamlining workflows. Findings 6: Redundancy is a practical maintenance issue in skill evolution. Developers can simplify skills by removing unnecessary artifacts, reducing duplicated instructions, and streamlining workflows.

Table II shows how developers modify skills during evolution. The most common change operation is Knowledge Enrichment (9.00%), followed by Workflow Refinement (8.50%), Traceable D. RQ4: Motivations of Skill Evolution Metadata Change (7.59%), Knowledge Modification (7.34%), We investigate the intentions behind skill evolution by and Workflow Extension (6.14%), together accounting for 38.57% of all labeled changes. In contrast, developers modify assigning a motivation label to each modified component skills through structural changes much less frequently. Knowl- (Section III-D), reporting both the top-level distribution and edge Restructuring (1.74%), Workflow Integration (0.07%), fine-grained categories. As shown in the pie part of Figure 8, Perfective maintenance and Artifact Refactoring (0.63%) together account for only 2.44% of all labeled changes. The relatively lower prevalence accounts for 50.37% of all changes, surpassing the sum of the of structural changes suggests that developers are more likely to other three types. This indicates that Perfective maintenance is improve skills through incremental edits than by reorganizing the dominant motivation for skill evolution. The fine-grained their overall structure. One possible reason is that such changes motivations further support this tendency. Specifically, Comare more likely to break or compromise skill functionality and pleteness Improvement is the most frequent fine-grained motivation, accounting for 15.61% of all changes, followed by Knowloften require more careful consideration. edge Improvement (9.76%) and Retrieval Improvement (9.38%), Findings 5: Change patterns are dominated by incremental which suggests that skills often require detailed task-related enrichment and refinement, while structural reorganization information and clearer retrieval cues after their initial release. is rare. One possible reason is that, unlike traditional software, skills

are expressed in natural language with inherently ambiguous correctness boundaries, encouraging developers to continuously refine their skills. For example, 0xBigBoss/skills/tilt added a “Verifying Updates Applied” workflow with shell commands to check resource status and wait until resources are ready, thereby improving the completeness of the update process. naksha-studio/skills/design expanded the trigger condition with visualization keywords, making the skill more likely to be invoked for visualization-related requests.

TABLE IV: Agreement between automated annotations and manual annotation labels. The first Component is sampled skills in RQ1.1, and the latter is the sampled skills in RQ2-4. Task

Unit

N

Component (Section III-C) Component (Section III-D) Change pattern (how) Fine-grained motivation (why)

Skill 383 Skill 27 Commit 404 Commit 404

Accuracy 0.881 0.913 0.816 0.840

V. D ISCUSSIONS Findings 7: Skill evolution is dominated by Perfective maintenance. This indicates that skills require continuous refinement to become functional, accurate, and invocable for agents after initial creation.

A. Automated Annotation Accuracy

Table III shows that only a small portion of Security Enhancement commits are accompanied by a reduction in high-or-critical risk findings, with an average reduction rate of 13.01% across the two scanners. In contrast, most commits remain unchanged, and a few commits are even associated with an increase in risk findings. This suggests that most security-related changes do not lead to security improvements observable by existing skill-security scanners.

Based on our empirical findings, we summarize the following research opportunities: Execution-contract skill design. RQ1 and RQ2 show that verification and recovery are both underrepresented in current skills and their evolution. This suggests that many skills still describe task procedures without specifying how agents should check results or handle failures. Due to the inherent ambiguity of skills as natural language artifacts and the stochastic nature of LLM-based execution, the outcome and procedure can be highly unpredictable. Future work could move beyond instruction-only skill design and define execution-contract schemas that include expected outputs, validation criteria, and recovery actions. Automating just-in-time skill updating. RQ1.2 shows that frequent skill evolution is concentrated in a small subset of complex and project-coupled skills. As these skills are closely

As described in Sections III-C and III-D, we first constructed the component taxonomy and the evolution pattern/motivation taxonomy based on manually inspected samples, and then used LLMs for large-scale automated annotation. To examine the reliability of the automated annotations, we report the TABLE III: Scanner Results for Security Enhancement commits manual annotation results for components, change patterns, and motivations, respectively, and cross-validate them against after manual filtering. the model-generated annotations. Due to space limitations, we only report accuracy scores. Detailed results are provided in Scanner Commits Decreased Unchanged Increased our online appendix1 . Cisco 876 133 (15.2%) 633 (72.3%) 110 (12.5%) For component annotation, accuracy is computed at the Snyk 876 95 (10.8%) 717 (81.8%) 64 (7.3%) Markdown-block level: we align blocks between the manual annotation and the model annotation within the same skill, Moreover, we observe that security-related motivations and compare their assigned component labels. For change appear relatively infrequently. To further contextualize this pattern and motivation annotation, we compute accuracy over category, we conduct an auxiliary analysis of changes labeled changed components within each commit. Specifically, we as Security Enhancement, examining whether these security- first align changed components between the reference and motivated commits coincide with observable reductions in model annotations. Then, for each aligned changed component, scanner-reported risks. We manually check the 1,156 Security the “how” or “why” prediction is counted as correct only Enhancement commits and retain 876 valid cases for analysis. when it matches the reference label. Changes associated with Specifically, we use two publicly available skill security unmatched components are treated as annotation errors. The scanners, Cisco Skill Scanner [43] and Snyk Agent Scan [44], reported accuracy is averaged over commits. As shown in Table IV, the component annotation achieves to scan for potential risks in skills. We analyze their outputs separately and use the number of reported high or critical risks an accuracy of 0.881, while the how and why annotations as the main comparison metric. For each commit, we compare achieve accuracy scores of 0.816 and 0.840, respectively. These this number before and after the change: a commit is classified results indicate high agreement between model annotations and as Decreased, Unchanged, or Increased when the number of human annotations, supporting the reliability of the large-scale high-or-critical risks becomes smaller, remains the same, or annotation results used in our analysis. becomes larger after the commit, respectively. B. Research Opportunities

Findings 8: Security-related skill changes rarely translate into observable scanner-confirmed risk reduction. This highlights a gap between developers’ security intent and actual security outcomes, underscoring the need for more effective automated tools to help developers make substantive improvements to skill security.

tied to project-specific contexts, they must be continuously to software ecosystems: from ad hoc scripts, to reusable updated to remain consistent with evolving codebases, depen- components, and eventually to maintained ecosystems. In dencies, and tool interfaces. Such continuous synchronization such an ecosystem, skill evolution is not simply editing can impose substantial maintenance costs when performed documentation, but the basis for keeping skills usable, reliable, manually. These findings highlight the need for automated and accountable as tasks, tools, and environments change. Our just-in-time skill updating mechanisms that can detect relevant study shows that current skill evolution is active but still project changes and adapt associated skills accordingly, thereby immature, with many changes focusing on refinement and reducing developers’ maintenance burden. completeness, while verification, recovery, long-term feedback, Context-efficient skill representation. Simplification-related and security remain underdeveloped. This suggests that future changes in RQ3 show that redundancy and accumulated skill ecosystems need maintenance infrastructure built around complexity are practical maintenance issues. At the same time, four properties: skills should be triggerable for correct retrieval, every loaded skill competes with task history, observations, tool attributable in terms of why they change, localizable when outputs, and retrieved documents for a limited agent context. failures occur, and traceable across their evolution history. By This motivates compression and refactoring techniques that characterizing what, how, and why skills evolve, our work remove redundant or irrelevant content while preserving the provides an empirical foundation for building such reliable and maintainable skill ecosystems. semantics required for reliable execution. Retrieval-aware skill engineering. Skill retrievability emerges VI. T HREATS TO VALIDITY as a recurring maintenance concern. RQ4 identifies Retrieval Improvement as one of the most frequent motivations. However, Internal Validity. Our component-level analysis relies on RQ1.1 shows that 28.9% of skills lack trigger conditions model-based identification of components and their subsequent and 7.5% lack functionality descriptions, which may limit “how”/“why” annotations. To mitigate this, we first constructed the precise retrieval of skills for a given task. Future work the component and evolution taxonomies from manually could build on our dataset to study automated techniques for inspected samples, used structured prompts with predefined improving skill retrievability, such as trigger-condition gener- label spaces, and evaluated automated annotations against ation, functionality-description refinement, retrieval-keyword manually labeled reference data. The annotation accuracy scores for component, change-pattern, and motivation labels are 0.913, suggestion, and related-skill linking. Context-alignment monitoring. RQ4 shows that a significant 0.816, and 0.840, respectively, suggesting that the automated portion of changes aims to align skills with evolving execution annotations are reasonably reliable. contexts, involving updates to knowledge, formats, interfaces, External Validity. Our dataset is collected from public GitHub workflows, and resources. This reflects the tight coupling repositories, and the findings may not fully generalize to between skills and their surrounding execution contexts, such private skills or enterprise settings. Nevertheless, GitHub is the as tools, data formats, and APIs. A promising direction is to largest open-source code hosting platform, and public agent monitor these contextual dependencies and detect when a skill skill repositories represent the mainstream practices of the becomes stale or inconsistent with the execution environment. developer community. In addition, the findings of RQ2–RQ4 Skill regression testing. The scanner-based analysis in RQ4 are based on relatively active and visible skills, and may not shows a gap between security intent and security effect: fully generalize to low-activity skills or skills from less visible security-motivated commits do not always reduce risks, and repositories. Nevertheless, skills with fewer evolution events some even increase them. This calls for regression testing provide limited evidence for fine-grained evolution analysis; methods that evaluate skill behavior before and after a change. focusing on actively evolved skills ensures sufficient change Possible tests include replaying representative tasks and check- samples for reliable component-level analysis. ing whether the success rate, safety, retrieval accuracy, or VII. C ONCLUSION execution cost degrades. This study systematically analyzes the evolution characPattern-guided skill enhancer. RQ4 shows that skill evolution is dominated by Perfective maintenance, indicating that many teristics of real-world agent skills. We present the current skills are functional but still incomplete, under-specified, or status of skills and conduct a fine-grained investigation along difficult to use effectively. This creates an opportunity for three dimensions: modified components, change patterns, pattern-guided skill enhancers. Such tools could diagnose and evolution motivations. Using data mining, open coding, weak components, such as missing trigger conditions, sparse semantic annotation, and statistical analysis, we identify several verification steps, incomplete recovery logic, or insufficient findings. Agent skills currently lack sufficient attention to domain knowledge, and recommend component-level edits reliability and failure handling in both their composition and evolution. Evolution changes concentrate in a small subset of based on common evolution patterns. project-coupled skills, while modified content skews toward C. Skill Ecosystem Evolution Trends instruction-oriented and perfective improvements. Change patAgent skills are moving from isolated markdown files terns are dominated by incremental enrichment and refinement, toward shared and maintained capability artifacts. Recent skill and redundancy emerges as a recurring maintenance issue. markets and community repositories suggest a trajectory similar Security-motivated changes rarely translate into observable risk

reduction, revealing a persistent gap between developer intent and actual security outcomes. These findings also motivate several future research opportunities: execution-contract skill design, automated skill update and refinement, skill regression testing, context-efficiency skill representation, retrieval-aware engineering, and context monitoring. By releasing our research dataset, future work could develop techniques for improving the quality, reliability, and maintainability of agent skills. VIII. DATA AVAILABILITY Our data, code, and online appendix are available at https: //github.com/ZJU-CTAG/SkillEMP. ACKNOWLEDGMENT This research is supported by the National Natural Science Foundation of China (No.92582107), the Fundamental Research Funds for the Central Universities (No. 226-2025-00067), and the University Grants Committee / Research Grants Council (General Research Fund 16210725 and Theme-based Research Scheme T41-517 25-N). R EFERENCES [1] T. Raheem and G. Hossain, “Agentic ai systems: Opportunities, challenges, and trustworthiness,” in 2025 IEEE International Conference on Electro Information Technology (eIT), 2025, pp. 618–624. [2] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022. [3] F. F. Xu, Y. Song, B. Li, Y. Tang, K. Jain, M. Bao, Z. Wang, X. Zhou, Z. Guo, M. Cao et al., “Theagentcompany: benchmarking llm agents on consequential real world tasks,” Advances in Neural Information Processing Systems, vol. 38, 2026. [4] X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang et al., “Agentbench: Evaluating llms as agents,” in International Conference on Learning Representations, vol. 2024, 2024, pp. 52 989–53 046. [5] C. S. Xia, Y. Wei, and L. Zhang, “Automated program repair in the era of large pre-trained language models,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), 2023, pp. 1482–1494. [6] C. Lemieux, J. P. Inala, S. K. Lahiri, and S. Sen, “Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), 2023, pp. 919–931. [7] S. Kang, J. Yoon, and S. Yoo, “Large language models are few-shot testers: Exploring llm-based general bug reproduction,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2023, pp. 2312–2323. [8] I. Bouzenia and M. Pradel, “Understanding software engineering agents: A study of thought-action-result trajectories,” in 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2025, pp. 2846–2857. [9] X. Du, L. Li, D. Zhang, and L. Song, “Memr 3 : Memory retrieval via reflective reasoning for llm agents,” arXiv preprint arXiv:2512.20237, 2025. [10] W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang, “A-mem: Agentic memory for llm agents,” arXiv preprint arXiv:2502.12110, 2025. [11] Z. Tan, J. Yan, I.-H. Hsu, R. Han, Z. Wang, L. Le, Y. Song, Y. Chen, H. Palangi, G. Lee et al., “In prospect and retrospect: Reflective memory management for long-term personalized dialogue agents,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 8416–8439. [12] Anthropic, “Equipping agents for the real world with agent skills,” https://www.anthropic.com/engineering/ equipping-agents-for-the-real-world-with-agent-skills, 2025. [13] W. Su, J. Long, Q. Ai, Q. He, Y. Tang, C. Wang, Y. Tu, Y. Wang, and Y. Liu, “Skill retrieval augmentation for agentic ai,” arXiv preprint arXiv:2604.24594, 2026.

[14] Y. Yang, J. Li, Q. Pan, B. Zhan, Y. Cai, L. Du, J. Zhou, K. Chen, Q. Chen, X. Li et al., “Autoskill: Experience-driven lifelong learning via skill self-evolution,” arXiv preprint arXiv:2603.01145, 2026. [15] C. S. Xia, Y. Deng, S. Dunn, and L. Zhang, “Demystifying llm-based software engineering agents,” Proceedings of the ACM on Software Engineering, vol. 2, no. FSE, pp. 801–824, 2025. [16] I. Bouzenia and M. Pradel, “You name it, i run it: An llm agent to execute tests of arbitrary projects,” Proceedings of the ACM on Software Engineering, vol. 2, no. ISSTA, pp. 1054–1076, 2025. [17] R. Xu and Y. Yan, “Agent skills for large language models: Architecture, acquisition, security, and the path forward,” arXiv preprint arXiv:2602.12430, 2026. [18] G. Ling, S. Zhong, and R. Huang, “Agent skills: A data-driven analysis of claude skills for extending large language model functionality,” arXiv preprint arXiv:2602.08004, 2026. [19] Y. Liu, W. Wang, R. Feng, Y. Zhang, G. Xu, G. Deng, Y. Li, and L. Zhang, “Agent skills in the wild: An empirical study of security vulnerabilities at scale,” arXiv preprint arXiv:2601.10338, 2026. [20] A. Zhao, D. Huang, Q. Xu, M. Lin, Y.-J. Liu, and G. Huang, “Expel: Llm agents are experiential learners,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, 2024, pp. 19 632–19 642. [21] S. Zhang, J. Zhang, J. Liu, L. Song, C. Wang, R. Krishna, and Q. Wu, “Offline training of language model agents with functions as learnable weights,” in Forty-first International Conference on Machine Learning, 2024. [22] Z. Z. Wang, J. Mao, D. Fried, and G. Neubig, “Agent workflow memory,” in International Conference on Machine Learning. PMLR, 2025, pp. 63 897–63 911. [23] Y. Zhou, W. Shu, Y. Su, W. Du, Y. Fang, and X. Lin, “A comprehensive survey on agent skills: Taxonomy, techniques, and applications,” arXiv preprint arXiv:2605.07358, 2026. [24] Y. Liang, R. Zhong, H. Xu, C. Jiang, Y. Zhong, R. Fang, J.-C. Gu, S. Deng, Y. Yao, M. Wang et al., “Skillnet: Create, evaluate, and connect ai skills,” arXiv preprint arXiv:2603.04448, 2026. [25] H. Li, C. Mu, J. Chen, S. Ren, Z. Cui, Y. Zhang, L. Bai, and S. Hu, “Organizing, orchestrating, and benchmarking agent skills at ecosystem scale,” arXiv preprint arXiv:2603.02176, 2026. [26] J. Ni, Y. Liu, X. Liu, Y. Sun, M. Zhou, P. Cheng, D. Wang, E. Zhao, X. Jiang, and G. Jiang, “Trace2skill: Distill trajectory-local lessons into transferable agent skills,” arXiv preprint arXiv:2603.25158, 2026. [27] Y. Zheng, Z. Zhang, C. Ma, Y. Yu, J. Zhu, Y. Wu, T. Xu, B. Dong, H. Zhu, R. Huang et al., “Skillrouter: Skill routing for llm agents at scale,” arXiv preprint arXiv:2603.22455, 2026. [28] D. Liu, Z. Li, H. Du, X. Wu, S. Gui, Y. Kuang, and L. Sun, “Graph of skills: Dependency-aware structural retrieval for massive agent skills,” arXiv preprint arXiv:2604.05333, 2026. [29] K. Zeng, Y. Huo, S. Zhang, Z. Ye, Y. Zhuo, H. Liu, Y. Lu, J. Wen, and X. Tang, “Group of skills: Group-structured skill retrieval for agent skill libraries,” arXiv preprint arXiv:2605.06978, 2026. [30] H. Pu, X. Song, and L. Zhao, “Skillops: Managing llm agent skill libraries as self-maintaining software ecosystems,” arXiv preprint arXiv:2605.13716, 2026. [31] H. Zhang, S. Fan, H. P. Zou, Y. Chen, Z. Wang, J. Zhou, C. Li, W.-C. Huang, Y. Yao, K. Zheng et al., “Coevoskills: Self-evolving agent skills via co-evolutionary verification,” arXiv preprint arXiv:2604.01687, 2026. [32] Z. Li and Y. Hu, “Skillhone: A harness for continual agent skill evolution through persistent decision history,” arXiv preprint arXiv:2606.08671, 2026. [33] Y. Liu, Z. Chen, Y. Zhang, G. Deng, Y. Li, J. Ning, Y. Zhang, and L. Y. Zhang, “Malicious agent skills in the wild: A large-scale security empirical study,” arXiv preprint arXiv:2602.06547, 2026. [34] GH Archive, “Gh archive: Github activity dataset,” https://www.gharchive. org, 2012, accessed: 2026-04-23. [35] Anthropic, “Agent skills specification,” https://github.com/agentskills/ agentskills, 2026, accessed: 2026-05-19. [36] H. Yang and H. Cai, “Dissecting real-world cross-language bugs,” Proc. ACM Softw. Eng., vol. 2, no. FSE, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3715777 [37] W. Charoenwet, P. Thongtanunam, V.-T. Pham, and C. Treude, “An empirical study of static analysis tools for secure code review,” in Proceedings of the 33rd ACM SIGSOFT international symposium on software testing and analysis, 2024, pp. 691–703.

[38] A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Ling et al., “Deepseek-v4: Towards highly efficient milliontoken context intelligence,” arXiv preprint arXiv:2606.19348, 2026. [39] E. B. Swanson, “The dimensions of maintenance,” in Proceedings of the 2nd International Conference on Software Engineering, 1976, pp. 492–497. [40] ISO/IEC/IEEE 14764:2022 Systems and Software Engineering – Software Life Cycle Processes – Maintenance, International Organization for Standardization Std., 2022. [41] S. Ouyang, J. M. Zhang, M. Harman, and M. Wang, “An empirical study of the non-determinism of chatgpt in code generation,” ACM Transactions on Software Engineering and Methodology, vol. 34, no. 2, pp. 1–28, 2025. [42] S. Vijayvargiya, X. Zhou, A. Yerukola, M. Sap, and G. Neubig, “Interactive agents to overcome ambiguity in software engineering,” arXiv preprint arXiv:2502.13069, 2025. [43] Cisco AI Defense, “Skill scanner,” https://github.com/cisco-ai-defense/ skill-scanner, 2026, accessed: 2026-06-15. [44] Snyk, “Agent scan,” https://github.com/snyk/agent-scan, 2026, accessed: 2026-06-15.

Related documents

• Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills#361525
• Subagents vs Agent Skills: Executing Reusable Knowledge for Long-Horizon Agentic Tasks#986999
• Towards Strategy-Level RSI for Skill-Augmented Agents: Learning When to Reuse Skills from Execution Feedback#1417941
• Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance#972728
• AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents#1297687