How Do Developers Maintain and Evolve Their Agents’ Instructions? An Empirical Study Gianmario Voria∗ , Alfonso Cannavale∗ , Andrea De Lucia∗ , Yutaro Kashiwa† , Gemma Catolino∗ , Fabio Palomba∗ ∗ University of Salerno, Italy
arXiv:2606.25257v1 [cs.SE] 24 Jun 2026
† Nara Institute of Science and Technology, Japan
Abstract—Context. Autonomous coding agents are increasingly used in software development, shifting parts of the engineering process to AI assistance. While this automation brings clear benefits, it introduces challenges in governance, traceability, and control over agent behavior. Agent Context Files (ACFs) have emerged as a practical mechanism to guide agents through structured instructions, yet little is known about how these artifacts are maintained and how their evolution relates to code development. Objective. This paper plans to investigate the evolution of ACFs and their role in agent-driven development. Specifically, we (1) classify ACF changes through a taxonomy grounded in software maintenance theory, (2) analyze how different types of changes are associated with code quality outcomes, and (3) examine their temporal patterns across the development lifecycle. Method. We conduct a large-scale mining study combining repositories with ACFs and agent-generated commits. We reconstruct ACF evolution at the commit level, classify changes using a qualitative approach, and analyze their association with code quality metrics. Statistical analyses and hypotheses are used to evaluate differences across maintenance categories, to inform future design of ACFs for governing autonomous coding agents. Index Terms—Coding Agents; Software Maintenance and Evolution; Empirical Software Engineering; Agent README.
I. I NTRODUCTION Autonomous coding agents are reshaping software engineering (SE) processes [1]. While practitioners traditionally managed the entire lifecycle, recent AI advances have shifted part of this responsibility to autonomous agents capable of executing complex tasks with minimal human intervention [2]. While automation has long been a central objective of SE research, historically focused on supporting developers in specific tasks, the current paradigm represents a substantial shift. Rather than assisting isolated activities, autonomous coding agents can orchestrate entire segments of the development lifecycle, introducing a new level of autonomy in software production [3]. This transition brings clear benefits in terms of efficiency and scalability, but also introduces new challenges related to control, transparency, and governance [4]. As development and evaluation activities become increasingly automated, greater attention must be devoted to upstream and downstream phases, particularly design and maintenance [5]. For instance, in requirements engineering, developers must now formulate specifications that are not only correct but also sufficiently precise and structured to guide autonomous agents, often in the absence of subsequent human interpretation or validation [6]. Similarly, in maintenance, the coexistence of
human- and agent-generated code introduces new complexities, including difficulties in tracing authorship and intent, understanding design decisions, and managing novel artifacts such as prompts and agent configurations [7]. One increasingly adopted artifact in this space is the Agent Context File (ACF) [8]. These artifacts can be seen as structured specifications that define how agents should behave within a project, including guidelines, constraints, and interaction rules. As such, ACFs act as an interface between human intent and agent execution, and have been proposed as a mechanism to reintroduce a degree of human governance over autonomous development processes [9]. While ACFs hold promise as governance artifacts, our understanding of how they are used and evolve in practice remains limited. In particular, while prior work has examined the effect of prompts and contextual information on agent behavior [10]–[12], considerably less attention has been devoted to understanding how these persistent governance artifacts are maintained over time and how their evolution relates to the evolution of the codebase. Studying the evolution of ACFs is important for several reasons. First, it provides insights into the challenges of instructing and controlling autonomous agents, as frequent or substantial changes may indicate misalignment between intended and actual behavior. Second, relating ACF changes to code evolution can reveal effective and ineffective practices, offering guidance for their design and maintenance. Finally, understanding when and how often ACFs are updated sheds light on how developers adapt governance mechanisms throughout the development lifecycle. Main Hypothesis. We hypothesize that Agent Context Files act as evolving governance artifacts in agentenabled software development, and that their modifications are not arbitrary but systematically reflect developers’ attempts to control and adapt agent behavior. In particular, we expect that ACF changes conform to classical software maintenance categories, and that different types of changes are associated with distinct code quality outcomes and exhibit different temporal and frequency patterns throughout the development lifecycle. To this end, we plan a large-scale mining study of reposito-
ries that adopt ACFs. We will extract commit-level information on ACF modifications and code changes, reconstruct their evolution over time, and derive an ACF-specific taxonomy of change types and interpret the resulting categories through the lens of classical software maintenance theory. We will then analyze these changes along three dimensions: (i) their distribution across categories, to assess the applicability of classical maintenance theory; (ii) their association with code quality outcomes through both structural metrics (complexity, coupling, size) and process-level indicators of corrective activity; and (iii) their temporal and frequency patterns within the development lifecycle. Beyond characterizing ACF evolution, our study is designed to yield actionable implications for both researchers and practitioners. For researchers, our taxonomy will provide a theoretically grounded framework for future studies on governance artifacts in agentic settings, while our findings on code quality outcomes will contribute empirical evidence on how the maintenance and evolution of persistent governance artifacts relate to agent-generated development outcomes. For practitioners, the findings can support the development of evidence-based governance practices for autonomous coding agents by identifying which types of ACF modifications are most common, when they are typically introduced, and how they relate to subsequent development outcomes. II. BACKGROUND AND R ELATED W ORK Autonomous Coding Agents in Software Engineering. The role of AI in software engineering has shifted from singleturn code completion toward fully autonomous agents that decompose high-level goals into sub-tasks, execute multi-step plans, and interact with tools such as test runners, debuggers, and version control systems with minimal human intervention [1], [13]. Wang et al. [13] characterize these systems through a perception–memory–action framework that highlights a form of explicit procedural memory encoded in the agent’s configuration, which governs how the agent perceives its environment and selects actions. This shift has enabled agents to take on increasingly complex SE activities, including automated program repair [14], autonomous testing [15], and agentic refactoring [16]. Greater autonomy, however, introduces governance challenges: empirical work has documented integration defects and security risks in agentic settings [17], and studies consistently identify the absence of authoritative project-specific knowledge as a primary failure mode [13]. Software Maintenance Theory. We ground our ACF evolution analysis in software maintenance theory (SWEBOK v4.0 [18]), which distinguishes between corrections (corrective, preventive, emergency) and enhancements (adaptive, perfective, additive). The operationalization of such classifications on version control data has been empirically demonstrated by Trautsch et al. [19], providing methodological grounding for our approach. Empirical Studies on Agent Context Files. To address the governance gap described above, practitioners
have adopted a class of structured Markdown files that encode persistent, project-level instructions for autonomous agents. These artifacts—named CLAUDE.md, AGENTS.md, or copilot-instructions.md depending on the tool— act as explicit procedural memory that developers write to shape agent behavior within a project, a mechanism that prior work has shown to affect generated code quality [11], [20]. Chatlatanagulchai et al. [8], [9] analyzed over 2,300 ACFs from 1,925 repositories, finding that these files are actively maintained through small incremental edits focused on functional concerns, with non-functional requirements rarely specified. Mohsenimofidi et al. [10] report no established structural standard across the 466 projects they examined. These studies confirm the practical relevance of ACFs but treat their evolution only marginally, without characterizing what changes are made, how they affect the codebase, or when they occur. Code Quality of AI-Generated Code. A growing body of empirical work has begun to examine the quality of code produced by LLMs and autonomous agents. Jamil et al. [21] compared GPT-3.5-Turbo and GPT-4 outputs against human-written solutions, finding that LLMs, when guided by advanced prompts, can outperform humans on several metrics. Studies using SonarQube to assess LLM-generated code have consistently found that functional correctness does not predict overall code quality [22]: even code that passes all unit tests still carries latent bugs, security vulnerabilities, and code smells. In agentic settings specifically, Horikawa et al. [16] found that agent-generated refactoring yields small but significant improvements in structural code metrics. Commit-Level Code Quality Measurement. Trautsch et al. [19] analyzed static metric changes across 54 open-source Java projects, classifying commits as perfective or corrective per Swanson’s taxonomy [23]. They found that perfective commits consistently reduce cyclomatic complexity, coupling, and code size, whereas corrective commits tend to increase them. We adopt their delta-based approach—computing metric differences between a commit and its parent—and apply it to agent-generated commits following ACF changes, using their findings to inform our hypotheses for RQ2. At the process level, Amit and Feitelson [24] proposed the Corrective Commit Probability (CCP), defined as the share of corrective commits within a development period. Validated on 7,557 GitHub projects with 93% accuracy, CCP correlates with larger files, higher coupling, and lower productivity, providing a reliable and language-agnostic proxy for code quality. We adapt CCP to the window level by computing it between consecutive ACF changes. III. E MPIRICAL S TUDY D ESIGN Our goal is to study how ACFs are maintained and evolve by deriving a taxonomy of their changes and analyzing their association with code quality and temporal patterns. A. Research Questions While ACFs are increasingly adopted to guide autonomous coding agents, their evolution in practice remains poorly
understood. Prior work has largely treated them as static specifications [8], [9], overlooking how developers iteratively refine these artifacts over time. In our first RQ, we will address this gap by grounding the analysis in software maintenance theory [18], treating ACF modifications as instances of established maintenance categories. Specifically, we will derive an ACFspecific taxonomy of change categories through qualitative analysis and subsequently examine how the resulting categories relate to established software maintenance concepts. RQ1 . To what extent can changes to Agent Context Files be characterized according to classical software maintenance categories, and how are they distributed? While characterizing ACF changes is a necessary first step, it is not sufficient to understand their role in the development process. As artifacts designed to guide autonomous coding agents, ACFs may shape how code is generated and evolves. Consequently, their modifications may alter agent behavior and be associated with measurable differences in the resulting code. Different types of ACF changes may reflect distinct intents and correspond to different outcomes in the codebase. Understanding these relationships is essential to assess the practical impact of ACFs, particularly in terms of code quality. To this end, in our second RQ, we will investigate whether different types of ACF modifications are associated with different code quality outcomes in subsequent code evolution, providing insights to inform their design and maintenance. RQ2 . Do different types of Agent Context File changes exhibit differences in their association with code quality outcomes in subsequent code evolution? While identifying the types of ACF changes and their association with code quality outcomes provides insight into what these changes are and what they do, it does not fully capture how they unfold throughout the development process. In particular, the timing and frequency of ACF modifications may reflect different maintenance intents and strategies adopted by developers when interacting with autonomous agents. From a software maintenance perspective, different categories of changes are expected to exhibit distinct temporal dynamics. For example, corrective changes may occur reactively in response to failures or undesired agent behavior, whereas perfective changes may be introduced proactively to refine or improve instructions [18]. Similarly, update frequency may indicate how actively developers adjust governance mechanisms over time. Understanding whether different types of ACF changes exhibit distinct temporal and frequency patterns is therefore essential to characterize how these artifacts are maintained, which will be the aim of our third RQ. RQ3 . Do different types of Agent Context File changes exhibit distinct temporal and frequency patterns throughout the development lifecycle?
B. Experimental Setup Datasets. We will rely on two complementary datasets to study the relationship between ACFs and software evolution. First, we will use the AIDev dataset [2], which provides a curated collection of repositories (116,211) and pull requests (932,791) involving agent-generated code. Agent contributions are identified through a validated classification pipeline, which we will adopt to extract commit-level information (e.g., file changes, diffs, metadata) for analyzing development activity. Second, we will use an existing dataset of repositories containing ACFs [8], comprising 2,303 context files across 1,925 repositories, including their content and versioning history. ACFs are identified through established naming conventions (e.g., AGENTS.md, CLAUDE.md, copilot-instructions.md) and represent persistent configuration artifacts used to guide agent behavior. We will reuse the original classification procedures where applicable. The study feasibility is supported by the original ACF dataset [8], containing over 10,000 ACF-modifying commits (for RQ1 ). For the remaining analyses, a preliminary pipeline extracted 10,763 commit snapshots with context files, 18,213 commits with file metadata, and 8,600 commits with intersecting information, confirming that sufficient data are available to conduct the study. Variables and Measures. The study will use two units of analysis: ACF-modifying commits (for RQ1 and RQ3 ) and development windows between consecutive ACF-modifying commits (for RQ2 ). Each ACF-modifying commit defines a window that includes all subsequent agent-generated code commits up to the next ACF change. These commits will be treated as outcomes associated with the preceding ACF modification. The ACF change category, once classified, will be used as the independent variable for all the comparisons. To address RQ1 , we will characterize ACF modifications according to software maintenance categories (corrections and enhancements) [18] and, more specifically, their subcategories (corrective, preventive, adaptive, perfective, and additive). This variable captures the nature of the change applied to the ACF and is used to analyze the distribution of maintenance categories and assess whether ACF evolution conforms to established maintenance patterns. To address RQ2 , we will measure code quality outcomes along two complementary dimensions. The first captures structural code quality through the delta of static metrics between each agent-generated commit and its parent, specifically cyclomatic complexity, lines of code, and coupling [19], [25]. We draw on Trautsch et al. [19] as a methodological precedent for measuring software quality evolution through changes in quality metrics. Unlike their study, which investigates the direct impact of maintenance activities on code quality, our analysis examines the association between ACF modifications and the quality of subsequent agent-generated code. Structural metrics are computed at the commit level and then aggregated at the window level. The second captures process-level quality through the Corrective Commit Probability (CCP) [24],
computed over the window of agent-generated commits between two consecutive ACF-modifying commits. Corrective commits are identified via a linguistic model applied to commit messages, making this measure fully language-agnostic and applicable across the heterogeneous repositories in our dataset. To address RQ3 , we will define variables capturing the timing and frequency of ACF changes throughout the development lifecycle. Specifically, we will consider relative timing, computed as the normalized position of a commit within the repository history, and change frequency, measured as the number of ACF modifications over a given number of commits or time window. These measures will allow us to analyze whether different types of ACF changes exhibit distinct temporal and frequency patterns. To account for potential confounding effects, we will consider additional variables at the commit, file, and repository level, including commit or ACF size, repository size, and activity level. Programming language will be recorded as a potential confounding variable and, where sample sizes permit, stratified analyses will be performed on the most represented languages. These variables will be included to isolate the relationship between ACF changes and the observed outcomes. C. Data Collection To investigate ACF evolution in actual code repositories, we will collect and combine commits from two publicly available datasets as described in Section III-B. ACFs Mining. We use the ACF dataset to identify repositories and commits that modify primary context files [9]. In this study, the term ACF refers exclusively to the primary repository-level context files identified in the original dataset (e.g., CLAUDE.md, AGENTS.md, and copilot-instructions.md). Auxiliary artifacts such as skills, imported context fragments, or task-specific instruction files are outside the scope of the analysis. We will consider all repositories in this dataset and treat each commit modifying a context file as a unit of change. We extract repositorylevel metadata and static representations of the context files, including identifiers (owner, name, URL), temporal metadata (creation date, first adoption), structural properties (e.g., length and complexity), and full textual content. To support longitudinal analysis, we will reconstruct ACF evolution by tracking commits that modify context files. For each, we will extract both the current and parent versions of the ACF, capturing modifications as before/after states. We also record commitlevel traces, including timestamps and change metrics (e.g., lines and structural changes), enabling fine-grained analysis of ACF evolution. Agentic Repositories Mining. We will integrate the ACF dataset with the AIDev dataset [2], which provides a largescale collection of agent-authored pull requests and commits. Since ACFs are available only for Claude Code, OpenAI Codex, and GitHub Copilot, we will restrict the analysis to these agents and retain only commits from repositories where a context file is present. Integration will be done at the repository level using canonical identifiers (owner/repository).
From the filtered AIDev data, we will construct a commitlevel dataset capturing structural and semantic properties of agent-generated changes. For each commit, we consider metadata (identifier, message, author, timestamp), size metrics (additions, deletions, total changes), file-level information (filenames and programming languages), and task-related annotations where available. Temporal information (e.g., timestamps and ordering) will be used to derive timing and frequency patterns of ACF updates throughout the development lifecycle. Additionally, we will implement the refined classification procedure proposed by Li et al. [2] to identify commits generated by specific agents. This procedure combines multiple signals, including author identifiers, commit message patterns, and naming conventions. While this approach provides a besteffort approximation, we acknowledge that agent attribution may not always be perfectly accurate. Commit Reconstruction. For each commit in the filtered dataset, we reconstruct a detailed snapshot of the development state. Specifically, we extract the full commit metadata and patch representation, enabling line-level analysis of changes. In addition, we retrieve all modified files in both the commit version and the immediate parent revision. For every commit, regardless of whether the ACF is modified, we will retrieve the version of the agent context file present in the repository at that specific point in time by querying the repository tree at the corresponding revision. This ensures that, for each commit, we can reconstruct the exact instructions available to the agent when producing changes. To support the measurement of structural code quality for RQ2 , the extracted diff and pre- and post-change file versions will be used to compute the delta in static metrics between each agent-generated commit and its parent. These per-commit deltas will then be aggregated over development windows defined by consecutive ACF-modifying commits. Data Quality and Filtering. We will include all commits from the filtered repositories that are associated with the selected agents. Repositories that are empty, inaccessible, or lack sufficient commit history will be excluded. A repository will be considered as adopting ACFs if at least one valid ACF is identified in its history and thus has been included in the original dataset [9]. For each commit, the presence of an ACF will be determined at the specific revision under analysis. Commits occurring before the introduction of an ACF in the repository will be retained but excluded from the main analyses, as no active governance artifact was in place at the time of their production. To ensure data quality, we will remove duplicated or malformed entries and verify the consistency of repository identifiers across datasets. All data and scripts will be made publicly available. D. Data Analysis To answer RQ1 , we will analyze all commits that modify an ACF to derive a taxonomy of ACF changes. Following Ralph’s recommendations for taxonomy development in qualitative software engineering research [26], we will employ an
inductive thematic analysis approach, as our goal is to identify and characterize recurring patterns of ACF evolution rather than develop a novel explanatory theory. We will first draw a statistically representative random sample of ACF-changing commits from the mined dataset. For each sampled commit, annotators will inspect: (i) the commit message, (ii) the ACF diff, (iii) the previous and current version of the ACF, and, when needed, (iv) the surrounding code changes in the same commit. The goal of this inspection is to identify the intent of the ACF modification. The analysis will proceed iteratively. Annotators will assign short descriptive codes to each ACF change (e.g., “clarify testing command” or “add coding convention”), reusing existing codes when appropriate and creating new ones when necessary. Through constant comparison and discussion, the codebook will be progressively refined and consolidated. Semantically related codes will then be grouped into higher-level themes representing recurring categories of ACF evolution. This process will continue until the taxonomy stabilizes and no substantial new categories emerge from the analyzed data. Because a single ACF modification may contain multiple logically distinct changes (e.g., removing obsolete instructions while simultaneously introducing new ones), annotators will assign multiple codes to the same modification if necessary. Each sampled change will be labeled independently by at least two annotators. Inter-rater agreement will be measured using Cohen’s κ. We will consider κ ≥ 0.70 as the minimum acceptable agreement level before proceeding with large-scale classification. If agreement falls below this threshold, the coders will refine the codebook and repeat the calibration process. To enhance rigor and trustworthiness, we will maintain an audit trail documenting coding decisions and taxonomy refinements throughout the analysis process. This process follows taxonomy-construction guidelines in software engineering [27], which require specifying the subject, descriptive basis, procedure, and validation strategy; here, these correspond to ACF changes, commit/ACF diffs, qualitative coding, and agreement-based validation. Once the initial taxonomy has been established, we will perform a deductive interpretation step to relate the identified categories to established software maintenance and evolution concepts [18]. Specifically, we will analyze how the emergent ACF change categories align with maintenance activities such as corrective, preventive, adaptive, perfective, and additive maintenance. The purpose of this step is not to guide the coding process, but rather to position the resulting taxonomy within the broader software engineering literature and enable comparison with prior studies on software evolution. After defining the entire taxonomy on the sampled data, we will classify the remaining ACF-changing commits. Depending on the dataset size, this step can be performed either manually or through an assisted procedure. For large datasets, we will adopt an LLM-as-judge approach, prompting the model with taxonomy definitions and labeled examples, and validating the resulting labels on an additional randomly sampled subset, following approaches used in prior studies
on ACF content [9]. The emergent ACF-specific categories constitute the primary outcome of the qualitative analysis. The maintenance-theory labels derived through the deductive interpretation step will subsequently be used as the analytical categories in RQ2 and RQ3 , enabling hypothesis testing, comparison across repositories, and alignment with established software evolution theory. To analyze the distribution of ACF changes across categories, we will follow statistical practices from empirical software engineering. As such data are often non-normal and heterogeneous [28], and RQ1 involves categorical counts, we will use a chi-square goodness-of-fit test to assess whether changes are uniformly distributed across categories [29]. We will complement significance testing with effect size reporting to quantify the magnitude of observed differences [28]. To analyze the distribution of ACF changes, we will test two complementary null hypotheses addressing the distribution of changes across (a) macro-categories and (b) subcategories. H1a There is no significant difference in the distribution of ACF changes classified as corrections versus enhancements. H1b There is no significant difference in the distribution of ACF changes across maintenance subcategories (corrective, preventive, adaptive-correction, adaptive-enhancement, perfective, and additive). If H1b is rejected, we will conduct post-hoc pairwise comparisons using proportion tests with Holm-Bonferroni correction. Rejecting any of these hypotheses would indicate that ACF evolution is not uniformly distributed and that certain maintenance types are more prevalent than others. In RQ2 , we plan to analyze whether different ACF changes are associated with differences in agent-generated code quality. For each commit N that modifies an ACF, we will consider the interval of commits between N and the next ACFmodifying commit M in the same repository as the ACF window. The window is therefore defined by ACF snapshots rather than temporal proximity. Autonomous coding agents interact with the current version of the ACF available in the repository regardless of when that version was last modified. Consequently, all agent-generated commits occurring between two consecutive ACF modifications are considered to operate under the same governance configuration and are associated with the same ACF snapshot. For the last ACF-modifying commit in a repository, the window extends to the last commit in the mined history. Each window will be associated with the category of the ACF change introduced in commit N , classified according to the taxonomy defined in RQ1 . Within each window, we will extract all agent-generated commits and exclude non-code artifacts. For each commit within the window, we will compute the delta of structural code quality metrics between the modified files and their previous versions. These metrics will then be aggregated at the window level (mean and median) to obtain a single representation of structural quality change for each ACF modification. In addition, we will compute the Corrective Commit Probability (CCP) over the same window. This captures process-level quality in terms of rework and defect-related activity.
To evaluate code quality across maintenance categories, we will employ the non-parametric Kruskal-Wallis test, followed by post-hoc pairwise Wilcoxon rank-sum tests with HolmBonferroni correction [28]. Effect sizes will be quantified via Cliff’s Delta. We test the following null hypotheses: H2a There is no significant difference in code quality outcomes between ACF maintenance subcategories. If H2a is rejected, we will test pairwise hypotheses: H2b-c-d There is no significant difference in quality outcomes between any pair of maintenance subcategories. In RQ3 , we will analyze when ACF changes occur during the development lifecycle and how frequently different types of changes are applied. We will define the lifecycle as the interval from the later of (i) the first detected ACF introduction and (ii) the first agent-generated commit, to the last mined commit. Within this interval, we will identify all ACFmodifying commits and compute their relative timing, operationalized through two complementary measures: (i) commitrelative timing, computed as the normalized position of the commit within the repository history, and (ii) time-relative timing, computed as the normalized position of the commit within the agent-enabled lifecycle in terms of elapsed time. We will measure the frequency of ACF changes for each maintenance category along two dimensions: the number of modifications per fixed number of commits (e.g., per 100 commits) and per unit of time (e.g., per day), capturing activity-relative and temporal update intensity. These measures will be computed per maintenance category to compare their temporal and frequency patterns. For statistical analysis, we will use the same procedure as in RQ2 . We define the following null hypotheses: H3a There is no significant difference in the temporal distributions of ACF changes across maintenance subcategories. H4a There is no significant difference in the frequency distributions of ACF changes across maintenance subcategories. If H3a or H4a is rejected, we will test the following pairwise hypotheses: H3b There is no significant difference in timing between any pair of maintenance subcategories. H4b There is no significant difference in frequency between any pair of maintenance subcategories. IV. T HREATS TO VALIDITY This section outlines potential threats to the validity of our study and the mitigation strategies that will be applied. Internal validity concerns whether observed relationships can be attributed to the studied factors. A key threat is linking code quality outcomes to ACF changes, as ACFs contain heterogeneous instructions and modifications may affect only parts of the file, limiting causal interpretation. To mitigate this, we adopt a window-based design and control for confounders (commit size, ACF size, repository size, activity level), framing results as associations rather than causation. Another threat concerns identifying agent-generated commits, which relies on heuristics; we mitigate this using validated procedures [2]. Lastly, because ACFs govern heterogeneous aspects of
development, some categories of modifications may have only indirect or limited influence on the selected quality measures, potentially reducing the strength of observable associations. Construct validity concerns how well measures capture the intended concepts. Code quality is multifaceted, and our use of structural and process-level metrics captures only part of it. We mitigate this by using complementary, literaturegrounded metrics and prioritizing language-agnostic measures. Another threat concerns the operationalization of temporal evolution. Relative commit position may not perfectly reflect lifecycle stage, so we employ two complementary timing measures: a commit-based relative position and a time-based relative position computed over the agent-enabled lifecycle. Differences in agent performance across programming languages may influence the observed outcomes. We mitigate this by explicitly recording language information and performing language-stratified analyses where feasible. Finally, classifying ACF changes may introduce subjectivity, as the taxonomy is derived through inductive qualitative analysis. Following established recommendations for rigorous qualitative software engineering research [26], we mitigate this threat through independent coding by multiple researchers, iterative codebook refinement, inter-rater agreement assessment, disagreement resolution procedures, and the maintenance of an audit trail documenting coding decisions. External validity concerns generalizability. Our study relies on publicly available repositories that adopt ACFs and autonomous coding agents [8], which may not fully represent industrial or proprietary settings. Moreover, the analysis is limited to agents for which both ACFs and agent-generated commits are available (Codex, Copilot, Claude). Conclusion validity concerns the reliability of statistical inferences. To mitigate statistical threats, we adopt established practices in empirical software engineering [28], including non-parametric tests, effect size reporting, and corrections for multiple comparisons, and we predefine hypotheses and analyses to ensure reproducibility. V. C ONCLUSION This report proposes a mining study on the maintenance and evolution of ACFs as governance artifacts for autonomous coding agents. By deriving an ACF-specific taxonomy and analyzing its relationship with code quality and temporal patterns, our findings are expected to inform both research and practice. For researchers, the taxonomy and empirical evidence provide a foundation for future work on governance in agentic settings. For practitioners, the results offer actionable guidance on when and how to evolve ACFs to guide autonomous coding agents effectively. ACKNOWLEDGMENT We gratefully acknowledge the financial support of JSPS KAKENHI (JP24K02921, JP25K03100), JST ASPIRE (JPMJAP2415), and JST CREST (JPMJCR23M1, JPMJCR26X7). We also acknowledge the use of GPT to ensure linguistic accuracy and readability.
R EFERENCES [1] A. E. Hassan, H. Li, D. Lin, B. Adams, T.-H. Chen, Y. Kashiwa, and D. Qiu, “Agentic software engineering: Foundational pillars and a research roadmap,” 2025. [Online]. Available: https://arxiv.org/abs/ 2509.06216 [2] H. Li, H. Zhang, and A. E. Hassan, “The rise of ai teammates in software engineering (se) 3.0: How autonomous coding agents are reshaping software engineering,” 2025. [Online]. Available: https://arxiv.org/abs/2507.15003 [3] X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large language models for software engineering: A systematic literature review,” ACM Transactions on Software Engineering and Methodology, vol. 33, no. 8, pp. 1–79, 2024. [4] A. Roychoudhury, “Agentic ai for software: thoughts from software engineering community,” 2025. [Online]. Available: https://arxiv.org/ abs/2508.17343 [5] C. Gao, X. Hu, S. Gao, X. Xia, and Z. Jin, “The current challenges of software engineering in the era of large language models,” ACM Transactions on Software Engineering and Methodology, vol. 34, pp. 1 – 30, 2024. [6] X. Lu, W. Sun, Y. Zhang, M. Hu, C. Tian, Z. Jin, and Y. Liu, “Requirements development and formalization for reliable code generation: A multi-agent vision,” Proceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 3932–3937, 2025. [7] M. Borg, D. Hewett, N. Hagatulah, N. Couderc, E. Söderberg, D. Graham, U. Kini, and D. Farley, “Echoes of ai: Investigating the downstream effects of ai assistants on software maintainability,” 2026. [Online]. Available: https://arxiv.org/abs/2507.00788 [8] W. Chatlatanagulchai, K. Thonglek, B. Reid, Y. Kashiwa, P. Leelaprute, A. Rungsawang, B. Manaskasemsak, and H. Iida, “On the use of agentic coding manifests: An empirical study of claude code,” in Proceedings of the 27th International Conference on Product-Focused Software Process Improvement (PROFES). Springer, 2025, pp. 543–551. [9] W. Chatlatanagulchai, H. Li, Y. Kashiwa, B. Reid, K. Thonglek, P. Leelaprute, A. Rungsawang, B. Manaskasemsak, B. Adams, A. E. Hassan, and H. Iida, “Agent readmes: An empirical study of context files for agentic coding,” 2025. [Online]. Available: https://arxiv.org/abs/2511.12884 [10] S. Mohsenimofidi, M. Galster, C. Treude, and S. Baltes, “Context engineering for ai agents in open-source software,” 2026. [Online]. Available: https://arxiv.org/abs/2510.21413 [11] N. Le Hai, D. M. Nguyen, and N. D. Bui, “On the impacts of contexts on repository-level code generation,” in Findings of the Association for Computational Linguistics: NAACL 2025, 2025, pp. 1496–1524. [12] Z. Chen and L. Jiang, “Evaluating software development agents: Patch patterns, code quality, and issue complexity in real-world github scenarios,” in 2025 IEEE international conference on software analysis, evolution and reengineering (SANER). IEEE, 2025, pp. 657–668. [13] Y. Wang, W. Zhong, Y. Huang, E. Shi, M. Yang, J. Chen, H. Li, Y. Ma, Q. Wang, and Z. Zheng, “Agents in software engineering: Survey, landscape, and vision,” Automated Software Engineering, vol. 32, no. 2, p. 70, 2025.
[14] I. Bouzenia, P. Devanbu, and M. Pradel, “Repairagent: An autonomous, llm-based agent for program repair,” in 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE, 2025, pp. 2188–2200. [15] R. Feldt, S. Kang, J. Yoon, and S. Yoo, “Towards autonomous testing agents via conversational large language models,” in 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2023, pp. 1688–1693. [16] K. Horikawa, H. Li, Y. Kashiwa, B. Adams, H. Iida, and A. E. Hassan, “Agentic refactoring: An empirical study of ai coding agents,” 2025. [Online]. Available: https://arxiv.org/abs/2511.04824 [17] Y. Shao, Y. Huang, J. Shen, L. Ma, T. Su, and C. Wan, “Are llms correctly integrated into software systems?” in 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE, 2025, pp. 1178–1190. [18] H. Washizaki, “Guide to the software engineering body of knowledge,” IEEE Computer Society, 2024. [19] A. Trautsch, J. Erbel, S. Herbold, and J. Grabowski, “What really changes when developers intend to improve their source code: a commitlevel study of static metric value and static analysis warning changes,” Empirical Software Engineering, vol. 28, no. 2, p. 30, 2023. [20] D. Shrivastava, H. Larochelle, and D. Tarlow, “Repository-level prompt generation for large language models of code,” in International Conference on Machine Learning. PMLR, 2023, pp. 31 693–31 715. [21] M. T. Jamil, S. Abid, and S. Shamail, “Can llms generate higher quality code than humans? an empirical study,” in 2025 IEEE/ACM 22nd International Conference on Mining Software Repositories (MSR). IEEE, 2025, pp. 478–489. [22] A. Sabra, O. Schmitt, and J. Tyler, “Assessing the quality and security of ai-generated code: A quantitative analysis,” 2025. [Online]. Available: https://arxiv.org/abs/2508.14727 [23] E. B. Swanson, “The dimensions of maintenance,” in Proceedings of the 2nd international conference on Software engineering, 1976, pp. 492–497. [24] I. Amit and D. G. Feitelson, “Corrective commit probability: a measure of the effort invested in bug fixing,” Software Quality Journal, vol. 29, no. 4, pp. 817–861, 2021. [25] T. J. McCabe, “A complexity measure,” IEEE Transactions on software Engineering, no. 4, pp. 308–320, 1976. [26] P. Ralph, “Toward methodological guidelines for process theories and taxonomies in software engineering,” IEEE Transactions on Software Engineering, vol. 45, no. 7, pp. 712–735, 2018. [27] M. Usman, R. Britto, J. Börstler, and E. Mendes, “Taxonomies in software engineering: A systematic mapping study and a revised taxonomy development method,” Information and Software Technology, vol. 85, pp. 43–59, 2017. [28] B. Kitchenham, L. Madeyski, D. Budgen, J. Keung, P. Brereton, S. Charters, S. Gibbs, and A. Pohthong, “Robust statistical methods for empirical software engineering,” Empirical Software Engineering, vol. 22, no. 2, pp. 579–630, 2017. [29] P. M. Bentler and D. G. Bonett, “Significance tests and goodness of fit in the analysis of covariance structures.” Psychological bulletin, vol. 88, no. 3, p. 588, 1980.