Understanding the Architecture of Coding Agents: An Exploratory Study Using a Research Prototype

Understanding the Architecture of Coding Agents: An Exploratory Study Using a Research Prototype Marco Tulio Valente Department of Computer Science Federal University of Minas Gerais, Brazil mtov@dcc.ufmg.br

arXiv:2608.10934v1 [cs.SE] 11 Aug 2026

Abstract Coding agents have rapidly emerged as the primary interface for AI-assisted software development. However, despite their growing adoption, relatively little is known about their internal architecture, and no systematic architectural description comparable to those available for compilers or operating systems currently exists. This paper addresses this gap by documenting the main architectural components of coding agents, explaining their responsibilities, interactions, and execution flow. To support this effort, we also present Ark (Agent Research Kit), a minimal open-source coding agent designed for research and education that preserves the essential architectural mechanisms of modern coding agents while emphasizing simplicity and clarity. We also introduce ArkBench, a lightweight benchmark comprising ten representative software maintenance and evolution tasks. Using gpt-5.4-mini, Ark successfully solved 8 of the 10 tasks while requiring modest token consumption. Finally, we compare the architecture of Ark with those of state-of-the-art coding agents using a recently proposed architectural taxonomy. We hope that both Ark and ArkBench provide a practical foundation for teaching, research, and experimentation on coding agents.

1

Introduction

Coding agents have emerged as one of the most important applications of Large Language Models (LLMs). Unlike first-generation AI programming assistants, which primarily answer questions or generate code snippets, coding agents autonomously carry out complete software development tasks through iterative interaction with an LLM and the local development environment. To this end, they can inspect source code, search for relevant information, run tests, edit files, and progressively refine their solutions until the requested task is completed. Consequently, coding agents are being rapidly integrated into the workflows of developers and organizations, becoming a core component of modern software engineering [8]. Despite their rapid adoption, relatively little is known about the internal architecture of coding agents. Commercial systems such as Claude Code and Cursor offer sophisticated capabilities but do not disclose their implementations, which makes it difficult to understand their internal design, reuse their architectural solutions, or adopt them as research platforms. Open-source alternatives, including Codex, OpenCode, and Aider, do make their source code available; however, these projects have already grown into large, complex systems comprising numerous modules and are designed primarily for production use rather than architectural clarity. Their documentation likewise focuses mainly on installation and usage, offering limited insight into the architectural principles underlying their design or the responsibilities of their core components. As a result, a systematic and clear architectural description of a modern coding agent is still lacking. This paper addresses this gap by documenting and explaining the main architectural components of coding agents. Specifically, we describe the responsibilities of these components, how they interact, and the execution flow that enables an agent to autonomously solve programming tasks. Our motivation stems from the rapid emergence of coding agents as the primary interface 1

for software development, much as compilers became indispensable with the advent of high-level programming languages in the 1970s. However, while compiler architectures have been thoroughly documented for decades [1, 4], no comparable architectural reference exists for coding agents. Another source of inspiration, from a different domain, was the Minix operating system, which was designed to make operating system architecture more accessible to students and researchers [11]. Furthermore, to ground our study in current practice, we implemented Ark (Agent Research Kit), a minimal, open-source coding agent designed for research and education. Rather than maximizing functionality, Ark emphasizes architectural clarity while preserving the essential mechanisms of modern coding agents, including the agentic loop, structured interaction with LLMs, tool invocation, agent memory, workspace management, and the safe application of code modifications. To evaluate Ark’s capabilities, we also developed ArkBench, a lightweight benchmark comprising ten representative software maintenance and evolution tasks, on which Ark successfully solved 8 of the 10 tasks. Finally, we compare the architecture of Ark with those of state-of-the-art coding agents using a recently proposed taxonomy for coding agent architectures [9], highlighting the architectural design choices that distinguish Ark from existing systems. The main contributions of this paper are fourfold: (1) a systematic description of the main architectural components of coding agents and the interactions among them; (2) Ark, a fully documented, open-source minimal coding agent designed as a platform for education, research, and experimentation; (3) ArkBench, a lightweight benchmark comprising representative software maintenance and evolution tasks; and (4) a practical foundation for future research on the design, implementation, and evaluation of coding agents. The remainder of this paper is organized as follows. Section 2 describes the methodology adopted to design and implement Ark. Section 3 presents the main architectural components of coding agents and explains how they are realized in Ark. Section 4 introduces ArkBench and reports the experimental evaluation of Ark. Section 5 compares the architecture of Ark with state-of-theart coding agents using a recently proposed taxonomy. Section 6 discusses related work. Finally, Section 7 concludes the paper and outlines directions for future research.

2

Methodology

To investigate the architecture of a coding agent, we implemented our own agent, called Ark. In the initial commits of this implementation, our goal was simply to follow the ReAct pattern proposed by Yao et al. [15]. During this early stage, we also relied on technical documentation published by leading AI companies, particularly an article by OpenAI describing the internal operation of Codex [7] and a similar article by Anthropic describing the architecture of Claude Code [3]. After studying the aforementioned references, the implementation was carried out iteratively by the author. Rather than developing all components at once, new functionality was introduced incrementally and validated through small-scale usage scenarios involving typical software engineering tasks, such as bug fixing, refactoring, and feature implementation. This iterative process made it possible to evaluate and refine the agent’s main architectural components, including the agentic loop, the interaction protocol with the LLM, agent memory, context management, and the available tools. During the early stages of development, two open-weight LLMs from the Qwen family (qwen2.5-coder:14b and qwen3-coder:30b) were used locally. However, their performance was generally unsatisfactory, particularly on tasks requiring strict adherence to the agent’s interaction protocol and reliable patch generation. Consequently, in the final stage of development, these models were replaced with a commercial LLM (gpt-5.4-mini), which exhibited more stable behavior and consistently better performance throughout our experiments. The development of Ark was itself supported by the Codex coding agent. In other words, we used one coding agent to implement almost the entire codebase of another. This decision provided valuable insights into the practical operation of coding agents. Nevertheless, all code generated by Codex was carefully reviewed by the author before being incorporated into the project. By the end of development, the project comprised 150 commits distributed across several categories. 2

As shown in Figure 1, 55 commits (37%) were devoted to refactoring the codebase, improving its modularity and readability. Another 48 commits (32%) focused on maintaining comprehensive and up-to-date documentation, particularly the README.md file. These results show that, as the architecture evolved, substantial effort was invested in simplifying the implementation and improving its documentation, consistent with the project’s goal of serving as a platform for research and education. The remaining commits correspond primarily to feature implementation (13%), bug fixes (7%), test development (5%), and configuration tasks (3%). In summary, Figure 1 shows that, although the code was generated by an LLM, considerable engineering effort, particularly through refactoring, was required to ensure high code quality. This was expected given the exploratory nature of the development process, in which the architecture gradually emerged through multiple rounds of trial and error. Refactor Docs Feature Bug Fix Tests 0

5

10

15

20

25 30 35 Number of Commits

40

45

50

55

60

Figure 1: Number of commits used in the implementation of Ark, by task category. Table 1 provides an overview of the source files that comprise Ark. As the table shows, the implementation meets its primary design goal of remaining compact and easy to understand. The entire codebase consists of just 13 Python source files totaling 1,471 lines of code (LOC). Furthermore, the project relies on only two external dependencies: openai, for accessing LLM APIs, and pytest, for running the test suite. Table 1: Main source files of Ark File agentic loop.py inputs.py models.py tools.py patches.py protocol.py finish handler.py traces.py

LOC 237 181 179 163 346 70 67 82

Description Implements the agentic loop and the agent’s memory. Loads configuration files and prompts, and prepares the workspace. Communicates with the LLM and processes its responses. Implements the tools available to the agent. Repairs, validates, and applies code patches. Defines and validates the interaction protocol with the LLM. Handles the finish action and coordinates final patch validation. Implements the tracing mechanism.

The codebase also includes a test suite comprising 50 unit tests distributed across eight test files. The tests were generated automatically with the assistance of Codex during development and subsequently incorporated into the project’s validation infrastructure. In the current version, all tests pass successfully, and line coverage over the Python modules in src/ark reaches 100%. Availability: Ark is publicly available, under a MIT license, at https://github.com/mtov/ark. .

3

Architecture of Coding Agents

Figure 2 illustrates the main architectural components of a coding agent and the relationships among them. As shown in the figure, the agentic loop plays a central role, coordinating the 3

interaction between the LLM and the other components. Before each interaction with the LLM, the agent constructs a context consisting of the system prompt, the task description, and the information stored in the agent’s memory. The LLM may then request the execution of a tool, whose output is recorded in the agent’s memory and incorporated into the context for the next iteration. This process repeats until the LLM determines that the task is complete and produces the final response. The following subsections examine each of these components in detail. For clarity, we first present the architectural role of each component and then describe how it is implemented in our reference implementation.

Agentic Loop

Tools

Task

System Prompt

LLM

Memory

Context

Figure 2: Architecture of a Coding Agent

3.1

Agentic Loop

The core component of a coding agent is the agentic loop. Through this loop, the agent repeatedly interacts with the LLM to solve a programming task. Rather than sending a single prompt and waiting for a final response, the agent and the LLM engage in multiple rounds of interaction until the requested task has been completed, as illustrated in Figure 3.

Task

Agentic Loop

LLM

Tools

Final Response

Figure 3: Agentic Loop Another important aspect is that coding agents typically follow the ReAct (Reason + Act) interaction paradigm [15, 6]. Under this paradigm, the LLM alternates between reasoning and 4

requesting the execution of actions. More specifically, the system prompt instructs the LLM to behave as follows: • The LLM must first reason about the problem. To do so, it decomposes the original task into smaller subtasks and addresses them one at a time. • The LLM may also request the execution of local programs, referred to as tools. Typical examples include listing, reading, searching, and editing files, running tests, executing Git commands, and applying patches. After executing a tool, the agent returns its output to the LLM. Based on this feedback, the LLM decides which subtask to perform next, which may involve requesting the execution of another tool. This process continues until the LLM determines that the original task has been completed successfully. At that point, the agentic loop terminates, and the LLM’s final response is returned to the agent and presented to the user. Because the agentic loop is the core component of a coding agent, Listing 1 presents its complete implementation in Ark. Despite its central role, the loop consists of only 18 lines of code. Execution begins by creating the agent’s memory (line 2), which records the tools executed during each iteration together with their outputs. The agent then enters a loop (line 4), bounded by a maximum number of iterations. In each iteration, the function get next tool request queries the LLM to determine the next tool to execute (line 5). If the requested action is finish, the agent returns the final response and terminates (lines 7–11). Otherwise, the requested tool is executed locally (line 15), and its output is recorded in the agent’s memory (line 16). If the iteration limit is reached, the agent terminates with an error (line 18). Listing 1: Agentic Loop 1 2

def agentic_loop ( config : AgentConfig ) -> LoopResult : memory = Memory ()

3

for iteration in range (1 , MAX_ITERATIONS + 1) : tool_request = g et_ne xt_t ool_r eque st ( config , memory )

4 5 6

if tool_request . name == " finish " : output = handle_finish ( config , memory , iteration , tool_request ) if output is None : continue return LoopResult . success ( output )

7 8 9 10 11 12

p r i nt _i ter at io n_a ct io n ( iteration , tool_request )

13 14

tool_result = run_tool ( tool_request , config ) memory . append ( iteration , tool_request , tool_result )

15 16 17

return LoopResult . max _i te rat io ns _re ac he d ()

18

3.2

System Prompt

LLMs typically receive two types of prompts: a system prompt and a user prompt. The system prompt provides high-level instructions that define the model’s behavior, including its role, constraints, and response format. The user prompt, by contrast, contains the specific request to be addressed, such as a question, a programming task, or a command. These prompts are passed as parameters in API calls to the LLM, as illustrated below: 1 2 3 4

messages = [ { " role " : " system " , " content " : " You are a coding assistant . " } , { " role " : " user " , " content " : " Fix the bug in add () . " } ]

5

The system prompt is therefore a fundamental design decision in a coding agent, as it defines the behavior the LLM must follow throughout its interaction with the agent. In particular, the system prompt should: (1) instruct the LLM to behave as a coding agent; (2) specify the interaction protocol, including the required response format; (3) describe the tools the LLM is allowed to invoke; (4) define operational constraints, such as permissions and file access; (5) guide the LLM’s decision-making process, for example, by specifying when it should read files, run tests, or conclude a task; and (6) define the termination criteria that determine when the LLM should produce the final response. Because of its central role, Listing 2 presents the complete system prompt used by Ark, which is stored in the file config/system prompt.txt. As shown, the system prompt is longer than the agentic loop itself, comprising 33 lines. It begins by defining the LLM’s role as a programming assistant (line 1) and specifying a ReAct-style interaction protocol, including the required response format and the actions—that is, tools—the LLM is allowed to invoke (lines 2–9). Next, it establishes rules governing tool usage and workspace access, restricting operations to the available files and discouraging redundant actions (lines 11–13). The prompt also imposes constraints to ensure consistent execution, such as requiring a file to be read before it can be modified and allowing the task to be completed only when sufficient evidence has been gathered (lines 15–23). Finally, it requires the final output as a diff patch. Listing 2: System Prompt 1 2 3 4 5

You are a coding assistant . Respond with exactly : Thought : your brief reasoning Action : list_files , read_file , find_text , run_tests , or finish Action Input : the file path , the search input , blank for run_tests , or a unified diff patch for finish

6 7 8 9

Choose exactly one action . Stop after that action . Never include Observation .

10 11

12

13

Treat the workspace root provided in the prompt as the only valid root for tool paths . Use only files and directories that were returned by tools or already appear in agent history . Do not invent paths or guessed roots such as ‘/ ‘ , ‘/ workspace ‘ , or ‘/ config ‘.

14 15 16

17 18 19

Do not repeat the same read or search unless you still need missing information . If you already read a file or ran the same search , use the earlier result from agent history and continue from it instead of repeating it , unless you need a different file or query . Prefer files and searches already listed in agent history . Before returning a patch , make sure you have read every file you are modifying . Do not edit , rename , or update a file that you have not read in the current run .

20 21

22 23 24 25 26 27 28 29

Do not use Action : finish before you have enough information from the workspace to complete the task . When you use Action : finish , return only a unified diff patch in Action Input . Do not claim that the patch was applied . Use real unified diff hunk headers with line ranges , for example : --- a / file . py +++ b / file . py @@ -1 +1 @@ - old_name + new_name

30 31 32

33

Use ‘ find_text ‘ with this format : search text | relative / or - known / workspace / path Use ‘ run_tests ‘ when the task asks you to validate behavior , fix behavior , or confirm a change . If workspace instructions are provided in the user prompt , follow them as local guidance for that workspace .

6

Designing the system prompt of a coding agent is largely an empirical process. For example, the first version of Ark’s system prompt consisted of only 13 lines and simply instructed the LLM to follow the ReAct paradigm while listing the tools it was allowed to invoke. As the agent evolved, the prompt was progressively refined. At one point, for instance, we observed that the LLM occasionally generated patches that modified files it had never read. This behavior increased the likelihood of inconsistent patches and application failures. To address this issue, the prompt was revised to explicitly require the LLM to read every file before proposing modifications to it (line 18 of Listing 2). Another important aspect is that the system prompt is sent to the LLM with every request issued by the agent. Consequently, its length directly affects token consumption and, therefore, execution cost and latency. For this reason, the system prompt should be kept as concise as possible without sacrificing the instructions essential for the correct operation of the agent.

3.3

Tools

One of the primary responsibilities of a coding agent is to provide the LLM with the information needed to complete the user’s requested task. To obtain this information, the agent executes local tools at the LLM’s request and returns their output to the model, as discussed previously. The tools that an LLM is allowed to invoke must be listed in the system prompt, as illustrated in Listing 2 (lines 4–5 and 31–32). In the current implementation of Ark, the following tools are available: • list files: Lists the files in a specified directory, allowing the agent to explore the project’s directory structure. • read file: Returns the contents of a specified file, enabling the agent to inspect the project’s source code. • find text: Searches for a given string within the files of a specified directory. It returns all matching occurrences, along with their file paths and line numbers, allowing the agent to locate code relevant to a particular task. • run tests: Executes the project’s test suite and returns its output, enabling the agent to verify whether the proposed modifications preserve the system’s expected behavior. • finish: Terminates the agent’s execution and returns a patch containing the modifications proposed by the LLM to implement the user’s requested task. In Ark, these tools are implemented directly in Python, which simplifies their integration with the other system components and provides greater control over the agent’s behavior. In other coding agents, however, tools may also wrap external programs, such as operating system utilities or version control commands like git.

3.4

Memory

A coding agent maintains a data structure, referred to as memory, that records the history of its execution and its interactions with the LLM. Depending on the implementation, this memory may include the task description, the responses generated by the LLM, the tools that have been executed, their arguments, and the corresponding outputs. In Ark, the memory primarily records the tools executed by the agent, the arguments passed to each tool, and the resulting outputs. As discussed next, this information is used to construct the context sent to the LLM in each new request and to prevent the agent from repeating actions that have already been performed. Figure 4 illustrates the classes used to implement Ark’s memory. The Memory class maintains a list of MemoryEntry objects, each representing one iteration of the agentic loop. Each entry records the iteration number, the executed tool (represented by a ToolRequest object), and the output 7

produced by that tool. Depending on the tool, this output may consist of the contents of a file, the results of a text search, or the output generated by executing the project’s test suite.

Memory

MemoryEntry

+entries: List<MemoryEntry> +append(...) +to_text() : : str

entries

+iteration: int +tool_request: ToolRequest +result: str

ToolRequest tool_request

+name: str +args: str

Figure 4: Classes implementing an Ark’s memory. In the current implementation of Ark, the agent’s memory is maintained entirely in main memory and is not persisted across executions. Consequently, each execution starts with an empty memory. This design choice simplifies the implementation and is well suited to scenarios in which executions are independent. However, it also prevents the agent from reusing information gathered during previous executions. Thus, a natural extension is to persist the agent’s memory to disk at the end of each execution, allowing it to be restored in future executions and introducing the concept of sessions. This mechanism would enable the agent to resume interrupted tasks and reuse knowledge acquired during previous executions.

3.5

Context

LLMs are stateless systems; that is, they do not retain information from previous interactions with a coding agent. Consequently, before each request, the agent must reconstruct and resend the information most relevant to the current execution, such as the tools that have been invoked and their corresponding outputs. Together, this information forms the context provided to the LLM, enabling it to determine the next step toward completing the task. In addition to the system prompt, every request sent by Ark to the LLM includes the following contextual information (see also Figure 5): 1. The description of the task to be performed, provided by the user in a prompt.txt file located in the project’s root directory. 2. Workspace-specific instructions, as defined in an AGENTS.md file located in the project’s root directory. When present, the contents of this file are incorporated into the context, allowing the agent’s behavior to be specialized for the particular project without modifying the system prompt. For example, an AGENTS.md file may describe project conventions, build and test commands, and other project-specific development guidelines [10, 5]. 3. An automatically generated summary of the agent’s current state, including the files that have already been read, the text searches that have been performed, and whether the test suite has been executed. This summary enables the LLM to quickly recover the overall execution state without reconstructing it from the interaction history described next. 4. A history of the agent’s most recent interactions, derived from its memory, including the tools that were executed, their arguments, and the corresponding outputs. Because LLMs have a limited context window, resending the entire execution history at every iteration is impractical. Instead, Ark includes only the four most recent iterations. This design decision reduces the number of tokens sent to the LLM, thereby lowering both cost and latency while still providing the information necessary for successful task completion.

8

User task: prompt.txt Workspace instructions: AGENTS.md Agent history • Files read: ... • Searches: ... • Tests: yes or no Recent steps • Iteration n • Tool: ... • Tool Args: ... • Observation: ...

Figure 5: Context information

3.6

Error Handling

The primary errors that a coding agent must handle arise during its interaction with the LLM. In Ark, two main classes of errors are addressed: inconsistencies in messages that are expected to follow the ReAct protocol and problems in the patches generated by the LLM. The following subsections describe how each of these error-handling mechanisms is implemented. 3.6.1

ReAct Protocol Errors

The Ark system prompt defines the fields that must appear in every response returned by the LLM (lines 2–9 of Listing 2). Specifically, responses are required to follow the ReAct format and include three fields: Thought, Action, and Action Input. The Thought field briefly records the reasoning behind the requested action. The Action field specifies the single tool to be executed by the agent. Finally, Action Input provides the arguments required to execute that tool, such as a file path, a search string, or a patch. The system prompt also explicitly states that responses must not include an Observation field. In ReAct-based agents, an observation corresponds to the output produced by the tool after it has been executed by the agent. In practice, however, LLMs do not always follow this protocol. For example, the model may omit the Action field, incorrectly include an Observation field in its response, or produce a free-form answer that does not conform to the required structure. Accordingly, Ark validates every response generated by the LLM. Whenever a response does not conform to the ReAct protocol, the agent returns the following prompt asking the LLM to correct its previous response: 1 2 3 4 5 6

Your previous response was invalid . Respond using only : Thought : ... Action : ... Action Input : ... Do not include Observation .

If the repair attempt also fails, Ark terminates execution and reports the error to the user. In the current implementation, only a single retry is performed. 3.6.2

Patch Errors

During the development of Ark, we observed that the final patch generated by the LLM was not always syntactically valid, even when the intended code changes were correct. For this reason, Ark 9

validates the patch before applying it. During this validation, the agent also automatically repairs minor inconsistencies that would otherwise prevent the patch from being applied successfully. A common example occurs when the LLM generates the correct patch content but reports incorrect line counts in the header of a hunk (a hunk is a contiguous block of modified lines within a patch). Suppose, for example, that the change simply replaces the line return 1 with return 2, together with three lines of surrounding context. If the LLM generates the hunk header as @@ -18,4 +18,4 @@ instead of @@ -18,3 +18,3 @@, the patch will be rejected, even though the intended modification is correct. In such cases, Ark automatically recomputes the line counts from the hunk contents and reconstructs the patch so that it can be applied successfully. Patch validation is performed using the git apply --check command, whereas patch application is carried out with git apply.

3.7

Security Mechanisms

As discussed in Section 3.3, coding agents may execute programs on the user’s local machine. Consequently, they must implement security mechanisms that restrict these operations and reduce the risk of unintended or potentially harmful modifications. In the current implementation of Ark, the following security mechanisms are provided: • Temporary workspace isolation. Ark never modifies the original project directory provided by the user. Before execution begins, it creates a copy of the project in a temporary directory named ark-workspace. All file reads, test executions, patch validation, and code modifications are performed exclusively within this temporary workspace. • Path restrictions. All file and directory access tools validate the requested paths to ensure that they remain within the temporary workspace. As a result, the agent cannot use these tools to read or modify arbitrary files on the host machine. • Restricted tool set. Ark exposes only a predefined set of tools to the LLM. Consequently, the model cannot request arbitrary operations on the local machine. • Fixed test command. The run tests tool does not allow the LLM to choose which testing framework or command to execute. Instead, Ark always runs the fixed command pytest, thereby reducing the risks associated with arbitrary command execution on the local machine. • Patch validation and user approval. Before any modification is applied to the workspace, the patch generated by the LLM is validated both syntactically and structurally, as described in Section 3.6.2. After successful validation, Ark displays both the command that will be executed and the proposed patch, requesting explicit user approval. Consequently, no changes are applied to the workspace without the user’s consent. Once approved, the patch is applied using the standard git apply command.

3.8

Tracing

Tracing is an important component of coding agents because it provides a record of the interactions between the agent and the LLM, as well as the actions performed while solving a task. Since the agent’s behavior emerges from multiple iterations, tool invocations, and intermediate decisions, tracing facilitate debugging, auditing, and failure analysis. In addition, tracing helps identify issues such as malformed LLM responses, protocol repair attempts, commands proposed to the user, and failures during patch application or test execution. In Ark, tracing is implemented through a plain-text file, agent trace.log, which records the requests sent to the LLM, their responses, validation errors, repair attempts, and events related to patch handling, including patch proposals, user approval or rejection, and final application.

10

3.9

LLM Integration

LLM integration is an essential component of a coding agent, as it is responsible for sending requests to the model and receiving its responses while abstracting the details of the underlying API. In Ark, this functionality is implemented by a dedicated module that loads the model configuration from a config.json file, sends system and user messages to the LLM, receives the model’s responses, collects token usage statistics, and records all interactions through the tracing mechanism. The module also allows users to configure the LLM provider, model name, and endpoint URL (provided it is compatible with the OpenAI API), and the environment variable that stores the API key.

4

Experimental Evaluation

This section presents the evaluation of Ark. We first introduce ArkBench, a lightweight benchmark specifically designed to assess coding agents on software maintenance and evolution tasks. We then report the results obtained by Ark on this benchmark and discuss its overall performance. Our primary goal with this benchmark is not to compare Ark with state-of-the-art coding agents. Instead, our focus is to demonstrate that the proposed architecture resulted in a functional prototype capable of solving tasks that emulate those performed by developers in their daily work.

4.1

ArkBench: A Lightweight Benchmark for Coding Agents

To evaluate Ark, we developed ArkBench, a small benchmark for coding agents focused on software maintenance and evolution. The benchmark was automatically generated by ChatGPT 5.5 from a set of design rules specified by the author, defining the application domain, programming language, project organization, task categories, complexity level, evaluation criteria, and coding style. The resulting source code, prompts, and tests were then manually reviewed to ensure correctness, consistency, and pedagogical quality. Unlike benchmarks based on large open-source projects, ArkBench is intentionally compact, inexpensive to execute, and easy to understand. Despite that, it provides realistic maintenance tasks that require an agent to understand an existing code base, locate relevant files, modify the implementation, and validate the resulting changes. The entire benchmark is implemented as clean, procedural Python code, avoiding unnecessary abstractions, external frameworks, databases, network communication, and accidental complexity. The benchmark is based on a small procedural e-commerce application. The system is organized into files responsible for product management, order processing, inventory, pricing, validation, filtering, formatting, and report generation, while data is represented using native Python structures such as dictionaries and lists. As summarized in Table 2, each file encapsulates a single well-defined responsibility, resulting in a compact and easy-to-understand code base that still requires agents to navigate multiple files during maintenance tasks. Table 2: Main files of the mini e-commerce application used in ArkBench. File

Responsibility

products.py orders.py inventory.py pricing.py validators.py filtering.py formatting.py reports.py

Product catalog management, including pagination and product search Order creation and processing Inventory availability and stock management Subtotal, discount, and total price calculations Validation of customer, product, and order data Filtering operations over product collections Formatting of monetary values and textual output Generation and export of sales reports

As presented in Table 3, ArkBench, contains ten independent maintenance tasks organized into three categories: three bug fixes, three refactorings, and four feature implementations. Each task

11

is self-contained, includes its own copy of the project, and requires modifications to approximately two to four source files. Table 3: Tasks included in ArkBench. Task

Description

bugfix-01 bugfix-02 bugfix-03

Fix an off-by-one error in catalog pagination Reject orders without a customer or products Fix the discount threshold calculation

refactor-01 refactor-02 refactor-03

Extract duplicated subtotal calculation into a shared function Move the price-formatting function to a dedicated file Rename a public product-search function and update its usages

feature-01 feature-02 feature-03 feature-04

Add product filtering by price range Add product sorting by price Export order reports in CSV format Group sales by product category

Each task follows the same directory structure. The file prompt.txt describes the requested maintenance activity in the style of a GitHub issue, without identifying the files to be modified. The tests directory contains the public test suite available to the agent, while the evaluation directory contains hidden post-task tests and, for refactoring tasks, structural verification scripts. These artifacts are executed only after task completion to validate additional scenarios and confirm that the requested refactoring has actually been performed. Bug-fix tasks contain at least one public test that fails in the initial version and passes after a correct implementation. Feature tasks start with all existing tests passing, while newly added hidden tests validate the requested functionality. Finally, refactoring tasks require all behavioral tests to pass both before and after the modification. In these cases, the scripts located in the evaluation directory additionally verify structural properties of the resulting code, such as the existence of a newly extracted function, the movement of a function between files, or the complete removal of an obsolete public API. Availability: ArkBench is publicly available at https://github.com/mtov/arkbench. .

4.2

Evaluation Results

Table 4 summarizes the results obtained by Ark on ArkBench. For each task, we report whether the task was completed successfully, together with the total number of tokens consumed, the execution time, and the number of tool invocations performed during the agentic loop. Overall, Ark successfully solved 8 out of the 10 benchmark tasks (80%). All four feature implementation tasks were completed successfully, as well as two of the three bug-fix tasks and two of the three refactoring tasks. The average execution time was 23.5 seconds per task, with most tasks completing in less than 30 seconds. On average, the agent consumed 3,342 tokens and performed 7.1 tool invocations per task, indicating that the benchmark can be executed efficiently while still requiring multiple interactions with the code base. The two unsuccessful tasks illustrate different challenges: • In bugfix-02, the agent produced only a partial repair. It correctly handled the invalidinput cases covered by the visible tests, allowing all public tests to pass. However, when the customer object lacked an id field, the patch still raised a KeyError instead of the expected ValueError. Thus, the issue was the use of a wrong exception type. • In refactor-03, the patch was applied successfully, but the rename was not propagated throughout the codebase. Although the agent updated the function definition and the visible 12

Table 4: Results of Ark on ArkBench. Task

Result

Tokens

Time (s)

# Tool Calls

bugfix-01 bugfix-02 bugfix-03 refactor-01 refactor-02 refactor-03 feature-01 feature-02 feature-03 feature-04

Success Fail Success Success Success Fail Success Success Success Success

2,430 4,176 2,315 4,024 6,584 4,461 2,422 2,207 2,441 2,360

18.04 63.94 14.84 24.14 18.99 23.16 26.73 14.19 15.75 14.94

6 8 6 8 10 9 6 6 6 6

Average

80%

3,342

23.47

7.1

unit test, it failed to update all imports and call sites, leaving one module dependent on the old name. Consequently, the public tests passed, but the hidden evaluation failed during test collection with an import error. This result illustrates a common limitation of LLM-based code agents in refactoring tasks: they often perform the primary transformation correctly while overlooking secondary dependencies, producing patches that are incomplete. The entire evaluation of Ark on ArkBench using gpt-5.4-mini cost less than US$0.50 in API usage. This result highlights the efficiency of Ark’s minimalist architecture: despite solving 8 out of 10 maintenance tasks, the agent required relatively few iterations, tool invocations, and tokens. It is also important to note that the results reported in Table 4 were obtained after a single execution of Ark. Thus, since LLMs are non-deterministic, they may vary across runs. Summary: We evaluated Ark using ArkBench, a lightweight benchmark comprising ten representative software maintenance and evolution tasks. Ark successfully solved 8 of the 10 tasks while maintaining low execution time, modest token consumption, and an API cost below US$0.50.

5

Comparison with Other Coding Agents

Recently, Rombaut proposed an architectural taxonomy for coding agents based on an analysis of 13 open-source systems [9]. The taxonomy organizes code agent architectures along twelve dimensions grouped into three layers. The first layer, Control Architecture, describes how an agent structures its execution flow and makes decisions. The second layer, Tool and Environment Interface, characterizes the agent’s interaction with the development environment, including the available tools, code editing mechanisms, and command execution facilities. Finally, the Resource Management layer describes how the agent manages internal resources such as state, context, memory, and language models. In this section, we use this taxonomy to compare the architecture of Ark with that of other coding agents. Rather than considering all the systems analyzed by Rombaut, however, we focus on two representative agents. The first is Codex CLI, developed by OpenAI, which represents the state of the practice in commercial coding agents. The second is OpenCode, one of the most popular open-source coding agents, representing projects developed and maintained by a broader community. The goal of this comparison is to position Ark within the architectural landscape of modern coding agents, highlighting its main design decisions and the simplifications introduced to make its architecture easier to understand and better suited for research and education.

13

5.1

Control Architecture

In Rombaut’s taxonomy, the Control Architecture layer describes how an agent organizes its execution flow, that is, how it decides which action to perform next and how it controls its interaction with the LLM. This layer comprises three dimensions: the type of control loop, the entity responsible for driving the loop, and the way the loop is implemented. Table 5 compares Ark with Codex CLI and OpenCode along these three dimensions.1 As shown, all three agents share the same control architecture. They implement a sequential ReAct-based loop in which the LLM itself decides, at each iteration, which tool should be executed next. Furthermore, all three rely on an imperative implementation based on a while loop to coordinate the successive interactions between the LLM and the available tools. These results show that, although Ark is substantially smaller and simpler than the other two agents, it adopts the same control architecture. Table 5: Control architecture.

Control loop Loop driver Control flow implementation

5.2

Codex CLI

OpenCode

Ark

ReAct Loop LLM-driven Imperative while loop

ReAct Loop LLM-driven Imperative while loop

ReAct Loop LLM-driven Imperative while loop

Tool and Environment Interface

This layer describes how the agent interacts with the local development environment. It encompasses the set of tools available to the LLM, the mechanism used to apply code modifications, how tools are exposed to the model, the mechanisms used for context retrieval, and the degree of isolation of the execution environment. Table 6 compares these characteristics across Codex CLI, OpenCode, and Ark. Table 6: Tool and environment interface. Codex CLI

OpenCode

Ark

Tool set size Edit and patch format

20+ Apply patch

5 Apply patch

Tool discovery Context retrieval Execution isolation

Per-turn rebuild Keyword/regex search Platform sandboxing

18+ String replacement and apply patch Dynamic Keyword/regex search Local shell

Static Keyword search Local shell

As shown in the table, the three agents differ substantially in their tool and environment interfaces. First, both Codex CLI and OpenCode expose more than 18 tools to the LLM. By contrast, Ark provides only five tools, as described in Section 3.3. This design choice reduces implementation complexity and makes the architecture more suitable for educational purposes. With respect to code editing, Codex CLI and Ark both adopt the apply patch mechanism based on unified diffs, whereas OpenCode supports both direct text replacement (string replacement) and apply patch. As discussed in Section 3.3, the apply patch approach facilitates patch validation before changes are applied to the repository. Another important difference lies in tool discovery. Codex CLI reconstructs the set of available tools dynamically at each iteration, allowing new tools to be incorporated while the agent is running. Similarly, OpenCode supports dynamic tool discovery through plugins and MCP servers. In contrast, Ark relies on a static set of tools defined when the agent is initialized, thereby simplifying the interaction protocol between the agent and the LLM. The agents also differ in their context retrieval mechanisms. Both Codex CLI and OpenCode support keyword and regular-expression 1

The Codex CLI and OpenCode columns in this and in the following tables are reproduced from Rombaut [9].

14

searches, providing greater flexibility for locating relevant code. In contrast, Ark implements only keyword-based search, which proved sufficient for the evaluation scenarios considered in this work. Future studies may, however, demonstrate the value of more powerful search capabilities. Finally, OpenCode and Ark execute commands directly through the local operating system shell, whereas Codex CLI relies on platform-provided sandboxing mechanisms to reduce the risks associated with executing harmful commands. This difference reflects Ark’s emphasis on architectural simplicity, whereas production-oriented agents incorporate more sophisticated isolation mechanisms.

5.3

Resource Management

This layer describes how an agent manages its resources during and across executions. In particular, it considers the representation of the agent’s internal state, the strategies used to reduce the size of the context sent to the LLM, the use of specialized language models, and the availability of persistent memory mechanisms. Table 7 compares these characteristics across Codex CLI, OpenCode, and Ark. Table 7: Resource management.

State management Context compaction Multi-model routing Persistent memory

Codex CLI

OpenCode

Ark

Flat list, preserved LLM-based summarization Safety-focused Background extraction pipeline

Typed event log LLM-based summarization Role-based Full session persisted in SQLite

Destructive Size truncation Single model (no routing) None

As shown in the table, the three agents differ substantially in how they manage internal resources. First, with respect to state management, Codex CLI maintains a linear history of messages throughout the session, whereas OpenCode uses a typed event log, in which events represent actions, observations, and state changes. By contrast, Ark adopts a simpler representation. As described in Section 3.5, its state consists only of the recent interaction history, with older entries discarded as needed. These differences are also reflected in the agents’ context compaction strategies. Both Codex CLI and OpenCode rely on the LLM itself to summarize the execution history when the context window approaches its limit. In contrast, Ark simply discards the oldest entries once the interaction history exceeds a predefined maximum size. Although less sophisticated, this approach is simple, predictable, and well suited to a minimal coding agent. Another important difference concerns the use of multiple language models. Codex CLI employs a specialized model to assess the risks associated with certain actions. Similarly, OpenCode supports model routing, allowing different models to perform specialized roles during execution. By contrast, Ark relies on a single model throughout the entire execution. Finally, the three agents differ in their support for persistent memory. Codex CLI stores relevant information in the background for reuse in future sessions. OpenCode persists the entire session state in an SQLite database, enabling previous executions to be resumed. In contrast, Ark does not retain any information across executions: its entire memory is discarded when the task completes. Summary: The comparison shows that Ark preserves the core architectural elements found in modern coding agents, particularly the ReAct-based control architecture driven by the LLM itself. The main differences lie in the Tool and Environment Interface and Resource Management layers, where Ark adopts simpler design choices, including a small predefined tool set, keywordbased context retrieval, the absence of persistent memory, and the use of a single model. These simplifications result in a smaller implementation that is easier to understand, modify, and extend.

15

6

Related Work

The work most closely related to ours is that of Rombaut [9], who proposed an architectural taxonomy for coding agents based on an analysis of 13 open-source systems. The taxonomy characterizes these agents along twelve dimensions organized into three layers, providing a comprehensive framework for describing the main architectural design decisions of modern coding agents. In this paper, we adopt this taxonomy as the basis for the comparative analysis presented in Section 5. However, whereas Rombaut focuses on comparing existing architectures, we present Ark, an open-source implementation that embodies the essential architecture of a coding agent and serves as a reference implementation for research and education. SWE-agent [14] was one of the first systems to demonstrate that coding agents can autonomously solve software maintenance tasks in real-world repositories, particularly issues involving bug fixes and small code modifications. The agent operates through a ReAct loop in which the LLM selects actions over an Agent-Computer Interface (ACI), such as navigating the repository, searching for information, editing files, and executing commands, while using the resulting observations to guide subsequent decisions. Its success influenced many later coding agents, helping establish the architecture based on an agentic loop and tool execution. Ark follows the same architectural model but places emphasis on architectural clarity and implementation simplicity. For example, 37% of its commits were devoted to refactoring aimed at making both the codebase and the overall design easier to understand. More recently, the authors of SWE-agent released mini-swe-agent, a reimplementation of approximately 100 lines of code intended to illustrate the core principles of the original system. Although remarkably elegant, this implementation abstracts away several components that are implemented in Ark, including memory and history management, patch validation and application. In other words, Ark was carefully designed to strike a balance between simplicity, architectural clarity, and completeness. AutoCodeRover [16] is a coding agent designed to autonomously resolve GitHub issues. Its distinguishing feature is the combination of LLMs with classical software engineering techniques, such as AST-based structural search and spectrum-based fault localization, to identify the relevant code before generating a patch. By contrast, Ark prioritizes a minimal, general-purpose architecture rather than incorporating specialized techniques for code localization and repair. OpenHands (formerly known as OpenDevin) is an open-source platform for research and development on software agents, designed to serve both as an execution environment and as an experimental infrastructure for agent-based systems [12]. The platform provides a comprehensive set of capabilities, including sandboxed command execution, code editing, web browsing, multiagent coordination, and an evaluation framework based on benchmarks such as SWE-Bench and WebArena. In contrast, Ark follows a deliberately minimalist philosophy. Rather than providing a comprehensive platform for developing and evaluating coding agents, its goal is to offer a small, educational, and easy-to-understand reference implementation of a coding agent’s architecture. Agentless [13] investigates whether coding agents based on iterative loops and dynamic tool use are truly necessary for solving software engineering tasks. Rather than adopting an agentic architecture, the system employs a fixed execution pipeline consisting of code localization, patch generation, and validation. Although this approach achieves competitive results on benchmark tasks, agent-based architectures have emerged as the dominant paradigm for programming assistants, demonstrating generality and flexibility to support a wide range of tasks. Two recent studies from our research group investigate coding agents from complementary perspectives. Santos et al. [10] analyzed 328 configuration files (CLAUDE.md) from public repositories, identifying the main software engineering concerns used to guide the behavior of coding agents, as well as the most common configuration patterns. Almeida et al. [2] evaluated GitHub Copilot Agent Mode for the automated migration of the SQLAlchemy library across ten real-world applications. Their results showed that, although the agent achieved high migration coverage, it still struggled to preserve the functional behavior of the migrated systems. These findings suggest several directions for future work with Ark. For example, it would be interesting to investigate

16

how project-specific configuration files, such as AGENTS.md and CLAUDE.md, affect the behavior and performance of coding agents. Another promising direction is to specialize Ark for specific software engineering tasks, such as automated code migration.

7

Conclusion

Coding agents are rapidly becoming a fundamental interface for software development. Despite their growing importance, however, their architecture has received relatively little attention from the software engineering community. This paper addressed this gap by presenting a systematic description of the main architectural components of coding agents, including the agentic loop, system prompt, tools, memory, context management, error handling, security mechanisms, tracing, and LLM integration. Our goal was to explain the architectural principles underlying modern coding agents and how these components cooperate to solve software engineering tasks. To make these concepts concrete, we presented Ark, a minimal open-source coding agent designed for research and education. We also introduced ArkBench, a lightweight benchmark comprising ten representative software maintenance and evolution tasks. The experimental results showed that Ark successfully solved 8 of the 10 benchmark tasks while maintaining low execution time, and modest token consumption. Finally, by comparing Ark with state-of-the-art coding agent, we showed that a minimal implementation can preserve the core architectural principles of modern coding agents while remaining smaller and easier to understand. Several directions remain for future work. First, Ark can be extended with capabilities commonly found in production-oriented agents, such as persistent memory, richer tool sets, more sophisticated context management, and multi-model routing. Second, ArkBench can be expanded with additional maintenance scenarios and larger projects to support more comprehensive evaluations. Finally, we hope that both Ark and ArkBench will serve as useful platforms for teaching, experimentation, and future research on the architecture and engineering of coding agents.

Acknowledgments This research is supported by grants from FAPEMIG and CNPq.

References [1] Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman. Compilers: Principles, Techniques, and Tools. Addison-Wesley, 2nd edition, 2006. [2] Aylton Almeida, Laerte Xavier, and Marco Tulio Valente. Using Copilot Agent mode to automate library migration: A quantitative assessment. In 1st International Workshop on Agentic Engineering, 2026. [3] Anthropic. How the agent loop works. https://code.claude.com/docs/en/agent-sdk/ agent-loop, 2026. [4] Andrew W. Appel. Modern Compiler Implementation in Java. Cambridge University Press, 2nd edition, 2002. [5] Hélio Victor F. dos Santos, Vitor Costa, João Eduardo Montandon, Luciana Lourdes Silva, and Marco Tulio Valente. Configuration smells in AGENTS.md files: Common mistakes in configuring coding agents. In 26th IEEE International Conference on Source Code Analysis and Manipulation, 2026. [6] Chip Huyen. AI Engineering: Building Applications with Foundation Models. O’Reilly Media, 1st edition, 2025.

17

[7] OpenAI. Unrolling the codex agent loop. unrolling-the-codex-agent-loop/, January 2026.

https://openai.com/index/

[8] Romain Robbes, Théo Matricon, Thomas Degueule, André Hora, and Stefano Zacchiroli. Agentic much? adoption of coding agents on GitHub. ACM Transactions on Software Engineering and Methodology, 35(1):1–31, 2026. [9] Benjamin Rombaut. Inside the scaffold: A source-code taxonomy of coding agent architectures. https://arxiv.org/abs/2604.03515, 2026. [10] Helio Victor F. Santos, Vitor Costa, Joao Eduardo Montandon, and Marco Tulio Valente. Decoding the configuration of AI coding agents: Insights from Claude Code projects. In 1st International Workshop on Agentic Engineering, 2026. [11] Andrew S. Tanenbaum. Operating Systems: Design and Implementation. Prentice Hall, 1987. [12] Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. Openhands: An open platform for ai software developers as generalist agents. In 13th International Conference on Learning Representations (ICLR), 2025. [13] Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Demystifying LLMbased software engineering agents. ACM on Software Engineering, 2(FSE), 2025. [14] John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, 2024. [15] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In 11th International Conference on Learning Representations (ICLR), 2023. [16] Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. Autocoderover: Autonomous program improvement. In 47th International Conference on Software Engineering (ICSE), 2025.

18

Related documents

• A Systematic Evaluation of Trajectory Data Curation for LoRA Fine-Tuning of Code Agents#386968
• Do Coding Agents Reuse Existing Code or Reinvent the Wheel?#1129949
• Do Coding Agents Reuse Existing Code or Reinvent the Wheel?#1108818
• Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025#635698
• Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025#633349