arXiv CSopen access

Projecting the Emerging Mindset of SWE Agent by Launching a Wild Code Understanding Journey

arXiv:2606.08500v1 [cs.SE] 7 Jun 2026

Springer Nature 2021 LATEX template

Projecting the Emerging Mindset of SWE Agent by Launching a Wild Code Understanding Journey Zhengyi Zhuo1 and Yan Liu1* 1

School of Computer Science and Technology, Tongji University, Shanghai, People’s Republic of China.

*Corresponding author(s). E-mail(s): [email protected];

Abstract Software engineering agents (SWE agents) are rapidly becoming popular in practice, yet we still know very little, in concrete, observable terms, about the behavioral regularities they exhibit when working in real codebases. Software engineering agents generate rich trajectories of tool use, intermediate reasoning, and self-directed stopping. This creates a basic contradiction: we know little about agent behavior, but we can collect abundant digital traces of it. The contradiction arises because trajectories record what the agent did, but not, by themselves, why those moves were chosen, what evidence was trusted, or what counted as sufficient understanding to stop. Still, precisely because the traces are faithful, replayable, and scalable, they offer one of the best empirical substrates for probing behavior and projecting the mindset underlying an agent’s reasoning and choices. To turn faithful traces into observable behavioral evidence, we build Ada as a scoped apparatus for repository-level code understanding and give it a world to move through. That world consists of real codebases made traversable through a bounded tool interface, so Ada’s exploration can remain open-ended in practice while becoming recordable as a finite trajectory. Inside this wild-but-bounded setting, Ada chooses where to look, what evidence to read closely, when to consolidate partial understanding, and when to close its account of the repository. These journeys produce think-action chains in which the progression from navigation through evidence selection and synthesis to stopping becomes visible as situated behavior. We project these chains through carefully designed lenses that offer usable distances for reading agentic behaviors, without reducing interpretations to a mechanistic accounting of tool calls or inflating them into speculative overreading of hidden intent. Read together, the views enable a

1

Springer Nature 2021 LATEX template 2

Projecting the Emerging Mindset of SWE Agent nascent projection of the SWE-agent mindset, expressed as behavioral profiles grounded in the agent’s recorded movement through real software worlds. Together, the apparatus, lenses, and 408-trajectory study provide a methodological foundation for observing SWE-agent behavior in real codebases, showing how faithful digital traces can be transformed into disciplined, comparable projections of emerging agentic mindset. The resulting observations align with broader findings on agent efficiency, trajectory diversity, epistemic grounding, and the limits of intervention, giving additional support to the proposed method beyond the study’s own experimental scope. Keywords: SWE agents, agent trajectories, repository exploration, code understanding

1 Introduction Autonomous agents, boosted by LLM reasoning engines, are moving fast into the open world, increasingly operating through trajectories of tool use, environment interaction, intermediate reasoning, and self-directed stopping. A valid, useful agentic interaction is no longer a single-turn final answer. It usually unfolds as a group of “turns” through a world, cascading the actions and reasoning into a continuous, visible flow. From the flow, every thought and action is seemingly easier to read closely, providing a translucent view of agent behavior. However, agent behavior does not automatically become legible from the thought–action chain. Recorded trajectories do not, by themselves, reveal the agent’s goals or strategic choices. Interpreting behavioral signals from outside the agent remains difficult, even though tools for logging and replaying agentic trajectories are mature and widely available [1–3]. Software engineering agents (SWE agents) make this challenge unusually concrete. Their world is digital by nature, organized into repositories whose structure spans files and symbols, cross-file dependencies, and executable behavior, all inspectable as the agent moves through them. It is open enough to require genuine navigation, yet bounded enough for the resulting work to be recorded and revisited. A high degree of digitization, however, does not by itself guarantee an environment that is legible from outside the agent. When an SWE agent enters a repository, it must form a working picture of a software system from partial observations. It must decide where to look, what to trust, when to consolidate, and when its understanding is sufficient to stop. These trajectories offer a rare site for studying agentic work as situated movement through a structured world. Interpreting the behavioral phenomena of an intelligent agent is pulled between two failure modes. At one extreme, a close reading of tool calls yields mostly mechanical signals rather than evidence of intelligent behavior; at the other, treating every move as motive imputes intention to every trace and therefore explains nothing. Still, an agent’s “mindset” is not beyond reach. Between these extremes lies a workable stance in which behavioral interpretations remain accountable to

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

3

APPARATUS / WORLD contains

LAUNCH Ada the SWE agent

(i)

produces

TRAJECTORY recorded trace

ThoughtActions

not mindset (solely)

RUNS / PLAYS here not mindset (by itself)

OBSERVERS

read through

PROJECTED MINDSET embodied in trajectories

(ii)

Analytical LENSES

equipped with

Fig. 1: Overview of the outside-in reading approach: an SWE agent interacts with a repository world through a tool-mediated apparatus, producing a recorded trajectory that is projected through observation lenses into comparable behavioral signals.

the traces that generated them. In this work, we develop a reproducible methodology for studying SWE-agent behavior from an analytical distance where recorded trajectories can begin to reveal an emerging mindset. Before any changes are made to the codebase, code understanding is the phase in which SWE agents (and also human developers) align their perceptions with the software system as it exists. This phase is especially revealing of mindset, because it is not reducible to task success: the agent must build understanding from partial evidence and then judge for itself when that understanding is sufficient to stop. A different line of work looks inside the model itself. Mechanistic interpretability, advanced most visibly by Anthropic, attempts to recover features and circuits from internal activations, offering a glass-box account of what the network represents. This access is real, but it depends on substantial interpretability infrastructure and compute. It also characterizes the foundation model more directly than the agent assembled around it. Our work starts from a complementary premise: even without opening the model, an agent’s “mindset” leaves systematic traces in its thought–action chain. Reading mindset from trajectories, therefore, offers a distinct scientific stance. Because it reads the assembled agent rather than the foundation model, this stance is tied to the artifact that developers actually ship. It is also comparatively inexpensive and suitable for comparison across models and conditions. At the same time, the limits of this outside view are not obvious. We treat the observability of mindset from traces as an open question, so our method probes the extent to which such readings can be inferred. As illustrated in Fig. 1, projecting mindset from outside requires (i) an environment that preserves genuine exploratory behavior while remaining bounded enough for repeated observation, (ii) analytical lenses that project recorded trajectories into comparable, interpretable views of how the agent oriented itself. This paper provides both. We build a minimal SWE agent apparatus that runs a multi-turn tool-use loop across six real-world repositories. A suite of observation lenses then renders different aspects of each trajectory into readable form. We use a dedicated code-understanding agent, Ada1 , as the subject of study. 1

Our code understanding SWE agent working inside the apparatus. Named after Ada Lovelace

Springer Nature 2021 LATEX template 4

Projecting the Emerging Mindset of SWE Agent

A recorded trajectory is not a direct readout of the agent’s internal mindset. It provides a faithful account of observable choices: where the agent looked, what it treated as evidence, how it consolidated partial understanding, when it decided to stop. Our method uses lenses to project those choices into a scoped behavioral profile. The projection supports disciplined comparison, without claiming access to hidden intent. Our contributions are as follows. 1. Within repository-level code understanding, we construct a bounded-yet-wild software world with scoped task activators. This setting makes the agent’s exploration recordable as trajectories that can be interpreted against the repository’s independently known structure. The world is designed as an observational instrument, providing a stable ground for making agent behavior legible and comparable. 2. At the level of trajectory interpretation, we formalize a suite of observation lenses that project recorded thought–action chains into structured views. These lenses provide usable analytical distances, from aggregate activity to temporal shape, action transitions, semantic grounding, and repository contact, without collapsing trajectory interpretation into raw tool-call accounting or superficial speculations. 3. For probing mindset observability, we operationalize a controlled launch-andintervention framework. Prompt-framing conditions expose how exploration changes under different evidential and resource constraints, while the coachjudge condition tests how far externally observed trajectories can be steered during a run. 4. Across 408 trajectories spanning multiple models, repositories, and launch conditions, we demonstrate that the apparatus makes SWE-agent code-understanding behavior empirically readable through its lenses. The projection lenses expose differences in navigation, synthesis, grounding, and stopping behavior. We also place these observations in dialogue with broader findings on agent trajectories, giving additional support to the proposed method. The remainder of this paper is organized as follows. Section 2 defines the SWE agent apparatus and formulates the research questions it enables. Section 3 describes the experimental scaffold, including repository-level code-understanding tasks, heterogeneous repository selection, foundation models and launching conditions, and the trajectory recording scheme, followed by the observation lenses through which recorded trajectories are projected into readable form. Section 4 reports results across three research questions, from surface signals through crosslens projection to the intervention boundary discovered under coaching. Section 5 situates the findings in the broader literature and discusses what the observation method makes transferable beyond this study. Section 6 addresses threats to validity and the epistemic boundaries of the apparatus. Section 7 surveys related work on trajectory-aware evaluation, tool-mediated interaction, and repository-level code understanding.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

5

2 Formulation of Research Questions 2.1 The Wild in Bounds: Crafting an Observable Digital World for SWE Agents Before asking how SWE agent mindsets can be projected from behavioral traces, we specify the object and setting that make such trajectories observable. We use the term SWE agent in a methodological sense. In this paper, an SWE agent is an LLM-mediated, tool-using interaction process. It is activated by a repository-level code-understanding task, acts through a finite sequence of turns in a bounded digital software environment, and terminates when it invokes a distinguished completion tool. We denote a scoped SWE agent as Aθ = ⟨Mθ , P, U, u∗ ⟩,

(1)

where Mθ is the foundation model; P is the prompt and role scaffold that defines the agent’s “framing” across launch conditions; U = {u1 , . . . , uk } is the set of available tools; and u∗ ∈ U is a designated termination tool whose invocation ends the run. Action selection follows the mechanism the foundation model implements given P , U , and the accumulated interaction history; we neither assume nor require an explicit, separable policy. A run is activated by placing this agent in a digital software environment E under a task τ and an experimental condition c: ρ = Run(Aθ , E, τ, c).

(2)

The run yields a finite trajectory: ξ = ⟨ (x0 , a0 , o1 ), (x1 , a1 , o2 ), . . . , xn ⟩,

n ≤ N,

(3)

where xt is the accumulated interaction history at turn t, at is the selected tool call, and ot+1 is the observation returned by the environment. The trajectory terminates when at = u∗ or when the turn bound N is reached. We define the surrounding apparatus as D = ⟨ W, T , R, L ⟩,

(4)

where W is the repository world, T is the set of task activators, R is the recording scheme that preserves each run as an analyzable trajectory, and L is a set of observation lenses. The repository world W consists of real software repositories that preserve their directory structure, cross-file references, and architectural layering, with evidence unevenly distributed across these structures. Task activators T launch the agent with directed but open-ended purposes that require repositorylevel understanding. The recording scheme R captures each run as a finite sequence of turns, preserving tool calls, observations, and intermediate reasoning artifacts within each turn.

Springer Nature 2021 LATEX template 6

Projecting the Emerging Mindset of SWE Agent Instrumented environment (bounded)

</>

– Agent

Codebase

Agent running logs

thinking • actions • responses

Recorded Trajectories Reasoning <thinking> contents

Bounded apparatus Environmental setup

files • code snippets • time

<- agent’s view

Navigation <tool> actions

Tool execution <system> results

derived information

observer’s view ->

Explored code areas

Time spent

coach condition

RQ1: Behavioral signals

The surface activities in statistics

RQ2: Mindset projection Under the suite of lenses

RQ3: Intervention boundary

Can intervention improve, or expose limits?

Fig. 2: Our bounded instrumented environment records both agent-understanding traces and environment traces, which are recomposed into observed trajectories that ground the paper’s three research questions on behavior, mindset projection, and intervention boundary.

In this paper, “mindset” refers to an externally projected stance that is read through the observation lenses. The emerging mindset is visible through the trajectories, by looking at them in a proper distance. The lenses create a spectrum of distances for the observers. Each observation lens ℓ ∈ L is a mapping: ℓ : Ξ → Vℓ

(5)

that projects the trajectory data into a readable view Vℓ , structured for comparison and interpretation, providing a scientific distance, namely a range within which we can observe and compare the agent behaviors under study. In this paper, we demonstrate five lenses: surface activity view, journey arc lens, action policy lens, automatic adjudication lens, and codebase footprint lens. These range from aggregate statistical summaries to fine-grained projections, both evaluative and spatial, of the trajectory. The set L is open; the five lenses demonstrated here are a minimal set sufficient to show that trajectory-level observation yields distinguishable behavioral signals. Researchers working with the same apparatus can define additional lenses over the same recorded data.

2.2 Research Questions Enabled by the Apparatus The apparatus above does not merely produce task answers. It produces observable trajectories: bounded records of how an SWE agent moves, reads, and reasons inside a repository world, how it stores intermediate understanding along the way, and how it terminates. This shift from final answers to trajectories motivates the research questions of this study. As illustrated in Figure 2, the bounded instrumented environment records both agent-understanding traces and environment traces, which are recomposed into

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

7

observed trajectories. These trajectories support three connected questions on behavior, mindset projection, and the intervention boundary. First, when an SWE agent is launched into a wild-but-bounded repository world, what behavioral signals are recorded, and what patterns emerge as the launch conditions change? Second, when those signals are interpreted through a mindset observation protocol, what do they reveal beyond aggregate statistical facts, and can we summarize them as behavioral profiles? Third, when an external observer is allowed into the agent channel through guided intervention, does the intervention improve the produced understanding, or does it instead reveal a boundary on what observation and control can perturb at the agentic behavioral level? RQ1: How do the signals gathered from SWE agent trajectories distribute within our wild-but-bounded environment? To explore whether the construction of our “wild digital world” supports robust observation, we launch Ada, the code-understanding SWE agent, in the experimental apparatus. The prompting status P was initially framed as the natural state, with no additional limits applied. And then, we used both budget pressure and prior suppression to stimulate the agent. Trajectories were recorded. RQ2: When projected through mindset observation lenses, what do these behavioral tendencies reveal of the SWE agent and the underlying LLMs? RQ1 asks whether the apparatus records behavioral signals at all. RQ2 asks whether those signals become meaningful only when read across multiple lenses, particularly in cases where surface statistics fail to explain process-level differences. Using the recorded trajectories from RQ1, we therefore interpret behavior through the ordered lens stack, moving from surface activity to temporal shape, action transitions, semantic adjudication, repository footprint. If trajectories can be observed from outside, the next question is whether that outside view can improve or steer the ongoing run. We treat this as an empirical question, not an assumption. RQ3: Under external observer-guided coaching, does the agent improve its answer, or does the intervention expose an intrinsic boundary of mindset? RQ2 motivates a third question. Can a trajectory-informed outside intervention steer the run toward better understanding, or does it instead expose a boundary on what external feedback can change? We connect the running agent to an online judging panel, then route panel feedback through a coach who decides when to intervene. In this setup, a run of Ada becomes: ρ = Run(Aθ , E, τ, ccj , C). (6) The trajectory produced by this run retains the sequential turn structure of Equation 3, but the content of each observation ot+1 is conditionally augmented. At each turn, a judge panel J reads the trajectory up to turn t and produces a diagnostic report J (ξ≤t ). The coach C receives the agent’s current action at and the panel’s report, and selects a decision dt ∈ {PASS, HINT, REJECT}:

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

8

dt = C at , J (ξ≤t ) . (7) The observation is returned to the agent, and subsequent actions depend on this decision.: ( rt+1 if dt = PASS, (8) ot+1 = rt+1 ⊕ mC if dt ∈ {HINT, REJECT}, where rt+1 is the tool execution result, and mC is the coach’s intervention message. Under PASS, the trajectory proceeds as in the main conditions. Under HINT, the coach provides strategic guidance alongside the tool result. Under REJECT, the coach returns the agent’s conclusion attempt with a diagnostic message, and the trajectory continues rather than terminating.

3 Methodology 3.1 Methodological Position: Projecting Code-Understanding Trajectories Before describing the experimental scaffold and the analysis protocol, we position the methodological stance that governs both. Repository-level code understanding does not terminate in an executable artifact or a pass-fail verdict. The trajectory of tool-mediated encounters with a repository is the primary empirical object, so the method must be designed around that object. Launching Apparatus PKR interaction

MEM

coach

tool action

HNT CON

advices

receive response

ASR Inference

Prompt Framing

BKM

Scoped Launch

reason

Architecture Analysis

EXP

Natural

Feature Tracing

External Intervention

Pressure Budget

CoachJudge

Prior Combined Suppression

Ada

Qwen3.5-397B-A17B

Action Policy Lens Lens 2

recording

Lens 0 Surface Activity View

ℛ schema

proper distance

Comparative Views

Data Visualization

Lens 1 Journey Arc Lens

Observer Gadgets

LM-enabled Semantic Lens

Automatic Adjudication Lens

Lens 3

Census Layer Analytical Statistics

DeepSeek-V3.2

GPT-4.1

trajectory logs

observe & read

mechanical view

GPT-4o

Hierarchical Lenses

termination (Player)

Flask requests scikit-learn prompt-optimizer Tensorzero graphiti

Risk Review

Models

Codebase References

f one o

The 6 Repositories

OVW

Conditions

Dependencies

Play Setups Tasks (Question Answering)

Narrative Layer

✓

Lens 4 Codebase Footprint Lens

! ×

Mechanism Layer

Evidence Layer

hyper-reflexive view

behavioral projections

SIGNALS | PROFILE | BOUNDARY

Fig. 3: Methodology overview. Within a bounded yet wild repository world, task activators launch Ada under controlled conditions, and the apparatus records its reasoning and tool-use trajectories. Observation lenses then project these trajectories into comparable views, supporting trajectory-level analysis of behavioral signals, mindset projection, and the identification of intervention boundary.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

9

In this paper, we treat trajectories as traces of observable choice, instead of direct access to internal mindset. The lenses operate as projections from those traces into scoped behavioral profiles. Trajectories do not interpret themselves. Aggregate statistics describe surface activity, but structured observation through the lenses formally defined in Section 2.1 is what makes behavioral profiles visible and comparable. Projection, as we use the term, extends beyond computing individual lens outputs. It includes reading across lens results under matched conditions, then anchoring the differences in trajectory-level evidence, before rendering the results visually so that aspects of an emerging code-understanding mindset become available for scientific discussion. Figure 3 previews the methodological position we take, including the observational distance at which trajectory reading becomes meaningful. Too close, a mechanical reading of raw tool calls yields activity without behavior, and signals that never resolve into a discussable profile. Too far, an over-interpretive reading turns every trace into motive and loses discriminability. Between these extremes lies a workable range in which interpretations stay accountable to the trace that produced them; we place our observer gadgets in this range, using bounded launches, trace-preserving recording, and lens-based projections to make agent processes observable without treating them as hidden internal state. The findings reported in this paper should be read as demonstrations that the method works within our bounded experimental environment. The apparatus is designed for structured observation, not competitive ranking. In this sense, Ada is not designed as a high-performance showcase or a benchmarking system for codereading performance. It is an observation instrument. By selecting and combining lenses, researchers can build observational instruments suited to viewing intelligent-agent trajectories at different levels of granularity.

3.2 Launching a Scoped SWE Agent into Code-Understanding Worlds We scope Ada so that its launches yield trajectories that are comparable across conditions and interpretable through our observation lenses. The scaffold fixes what can be recorded and keeps the account external. Repository access is read-only, so trajectories reflect how the agent reads the world rather than how it modifies it. A small typed tool surface further separates navigation from cognitive bookkeeping, making both observable in the recorded trace. Ada instantiates the scoped SWE agent Aθ = ⟨Mθ , P, U, u∗ ⟩ (Equation 1). We vary Mθ across the models in Section 3.3; the tool set U is summarized in Table 1; and termination u∗ is implemented by stash conclusion. The scaffold P follows a ReAct-style [4] design, informed by publicly available SWE agent prompt architectures [5]. Each turn externalizes a brief pre-action rationale and issues exactly one tool call, yielding a discrete sequence that can be compared across runs. The scaffold also carries forward an accumulating working state (memories, bookmarks, and conclusions), so trajectories show how local

Springer Nature 2021 LATEX template 10

Projecting the Emerging Mindset of SWE Agent

Table 1: Ada tool set, grouped by category. Navigation tools interact with the repository; cognitive tools manage the agent’s working state; the oracle tool requests researcher clarification; and the termination tool (u∗ ) records conclusions and governs session closure. Abbr.

Tool

Navigation tools OVW get structural

Usage

overview

Retrieve the high-level structural outline of a file or a named logical block within it.

PKR

peek code region

Read a specified range of source lines for detailed examination.

EXP

explore code

Navigate code relationships: jump to a symbol’s definition, find all references to a symbol, or advance to the next logical block.

Cognitive tools MEM create memory BKM

bookmark

Oracle tool HNT get hint on snippet

Termination tool (u∗ ) CON stash conclusion

Record a titled working note synthesizing current understanding, hypotheses, or partial models. Mark a code location with an attached question or reminder for later return.

Request a targeted clarification from the researcher on a code snippet that remains unclear after self-directed analysis.

Record a formal conclusion about the code and signal whether to continue analysis or finalize the session.

evidence is consolidated into intermediate understanding over time. Appendix A details the prompt template and interface. The tool set U defines Ada’s bounded interface to the repository world (Table 1). We choose a small set of typed, developer-like reading actions to make repository contact observable and comparable, while preventing uncontrolled side effects from arbitrary filesystem access. The tools separate navigation (where the agent looks) from cognitive bookkeeping (what it commits as intermediate understanding), a distinction that later lenses use to relate movement through the codebase to stabilization of beliefs. Termination is designed as part of the observation record. A run ends when the agent submits a concluding account via stash conclusion (the distinguished tool u∗ in Equation 1), or when the predefined turn-bound N is reached; separating intermediate synthesis from explicit closure keeps both analyzable in the trajectory. Before scaling the experiment, three prompt-scaffold variants were tested under the same tool design. All produced acceptable code-understanding accounts and reached explicit termination, confirming that the apparatus supports finite, recordable trajectories. The stricter (N = 100) turn bound was applied later in the main matrix. Detailed pilot materials are reported in Appendix B.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

11

3.3 Experimental Matrix: Tasks, Repositories, Models, and Conditions Repository-level code understanding unfolds through trajectories, and trajectories become scientifically comparable only when the launch conditions that produce them are specified across fixed axes. The experimental matrix varies over four axes over the constant scaffold described in the previous subsection: task family, repository, foundation model, and experimental condition. This structure makes the resulting trajectories part of a single observation design rather than isolated demonstrations. Behavioral signals become visible only when runs can be aligned along shared launch axes. Tasks provide the most natural separators of initial “kick-offs” on an agent’s journey, while prompt-framing conditions introduce controlled constraints that scope how the foundation model justifies claims and decides to commit to them. The coaching condition then moves from agent-internal setups to intervention by admitting an external observer into the agent channel and testing the boundary between what can be read from traces and what can be steered. Taken together, the matrix is deliberately scoped. It covers factors that SWE agent designers plausibly care about while avoiding implementation-level variations. The task family axis T specifies the initiating forces that bring the agent into a code-understanding situation, while leaving the ensuing route of exploration under agent control. We draw four task families from recognizable software-engineering situations (Table 2) [6] so that trajectories can be compared across distinct kinds of understanding work rather than a single prompt template. The repository axis W specifies the codebase worlds into which the agent is launched (Table 3). We intentionally mix system types, scales, and organizational styles so that observed behaviors are not artifacts of one architectural idiom, but stable enough to support cross-repository comparison. Varying the foundation model Mθ provides a comparison axis while holding the scaffold fixed, making differences in evidence use, exploration scope, and conclusion grounding observable in trajectories. The main experimental matrix instantiates Ada with four foundation models (GPT-4o2 , GPT-4.13 , DeepSeek-V3.24 , and Qwen3.5-397B-A17B5 ), chosen to provide bounded diversity rather than exhaustive coverage. Models that could not reliably sustain the tool-call protocol in pilot runs were excluded from trajectory generation.6 For readability, we later refer to DeepSeek-V3.2 as DeepSeek and Qwen3.5-397BA17B as Qwen, unless otherwise specified. GPT-4.1 and GPT-4o are kept as-is. The condition axis c varies prompt-framing conditions over the same scaffold, so cross-condition comparisons isolate changes in guidance and evidential stance rather than changes in tools or repository access. Budget pressure operationalizes 2

OpenAI model documentation: https://developers.openai.com/api/docs/models/gpt-4o. OpenAI announcement: https://developers.openai.com/api/docs/models/gpt-4.1. Model card (PDF): https://fe-static.deepseek.com/chat/transparency/deepseek-v3.2-model-card-0414-EN.pdf. 5 Model card: https://huggingface.co/Qwen/Qwen3.5-397B-A17B. 6 The GLM family was excluded from trajectory generation due to unreliable tool-call compliance, and was used only in the post-hoc evaluation layer. See Appendix B. 3 4

Springer Nature 2021 LATEX template 12

Projecting the Emerging Mindset of SWE Agent

Table 2: Task families and the software-engineering situations they reflect. Each family targets a distinct form of repository-level code understanding, grounded in a recognizable moment in real software work. Task Family

Software-Engineering Situation

Code-Understanding Challenge

architecture analysis

Onboarding, dependency evaluation, migration planning

Identify major components, architectural style, and interaction patterns across the repository.

ASR inference

Architectural reverseengineering in underdocumented or legacy codebases

Reconstruct implicit architectural requirements from code evidence.

feature implementation tracing

Bug localization, changeimpact analysis, feature-level debugging

Locate and describe the end-to-end implementation path of a specified feature.

risk and review

Code review, maintenance triage, technical-debt assessment

Identify risks, comprehension difficulties, or code smells with supporting code locations.

code-smell

Table 3: Repository fields used in the Ada apparatus. Repository

Language

Files

LOC

System Type

Requests Python 29 8,060 HTTP client library Role: Tests implementation tracing behind a focused public API. Flask Python 29 9,526 Web framework Role: Tests framework abstraction, context mechanisms, routing, and extension structure. Scikit-learn Python 554 258,342 Scientific ML library Role: Tests structural compression across a broad algorithmic ecosystem. Graphiti Python 109 14,321 Agent-memory infrastructure Role: Tests graph, LLM, retrieval, and service coordination. Prompt Optimizer TypeScript 172 30,694 Prompt-engineering product platform Role: Tests frontend, service layer, template, storage, and workflow integration. TensorZero Rust 438 149,755 LLMOps platform Role: Tests provider abstraction, inference, observability, evaluation, and optimization loops.

resource-aware launching to test how constrained planning reshapes trajectory organization [7, 8]. Prior suppression enforces an evidence-first stance to separate trace-grounded understanding from prior familiarity, without attempting to remove knowledge from the model in the sense of unlearning [9]. Finally, the coach-judge condition introduces an external observer into the agent channel to probe the boundary between what can be read from trajectories and what can be steered. Table 4 summarizes the conditions and their observation roles (the core grid yields |W| × |T | × |Mθ | × |c| = 6 × 4 × 4 × 4 = 384 trajectories). The four prompt-framing conditions in the core grid are summarized in Table 4 and provide the main comparison set for RQ1 and RQ2.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

13

Table 4: Experimental conditions. The core grid (top) systematically varies budget and prior-knowledge constraints across all four models and six repositories. The intervention condition (bottom) introduces an external coaching mechanism and is used exclusively for RQ3. Condition

What’s Different

Prompt-framing conditions (RQ1, RQ2) natural condition Standard launch; no special settings applied

Observation Purpose

Provides the unperturbed reference trajectory against which all other conditions are compared.

budget pressure

Explicit 12-turn budget awareness with progressive convergence reminders

Tests whether trajectory organization restructures when exploration space is compressed.

prior suppression

Evidence-first constraint treating prior familiarity as weak intuition rather than sufficient ground

Makes the boundary between priordriven and trace-driven reasoning more observable.

combined condition

Budget pressure and prior suppression applied together

Provides an interaction reference for observing both constraints in the same trajectory.

External intervention condition (RQ3) (only for GPT-4.1) coach-judge An external coaching and judging mechanism intervenes during the agent’s reasoning process

Tests whether external intervention improves or degrades what the agent’s mindset produces.

The four main conditions keep each trajectory self-directed. The coach-judge condition crosses that boundary by introducing an external coach whose message can enter the agent’s observation stream, based on a judge panel reading of the trajectory so far. The intervention mechanism is defined in Equation 6 (Section 2.2), and the pipeline details are given in Appendix A.2. The experimental matrix, therefore, defines the apparatus’s bounded empirical range. The 384 main-matrix trajectories and 24 coach-judge trajectories together constitute the 408-trajectory analysis object from which the observation lenses, introduced next, construct behavioral profiles.

3.4 Analysis Lenses: A Way of Seeing Figure 3 summarizes the analysis stack that connects raw trajectories to the claims reported in this paper. The observer gadgets provide the intermediate instrumentation: they operationalize aspects of agent behavior and transform long, toolmediated traces into stable, comparable views that support systematic reading across models, prompting conditions, repositories, and task families. Concretely, an observation lens ℓ ∈ L is defined as a map from trajectory evidence to a structured view, ℓ : Ξ → Vℓ (Section 2.1). The lenses are designed for empirical analysis, i.e., for researchers who must compare and aggregate behaviors across many runs rather than interpret individual trajectories in isolation. We instantiate five lenses as a working set; together they suffice to demonstrate that trajectory-level observation yields distinguishable behavioral profiles across models and conditions.

Springer Nature 2021 LATEX template 14

Projecting the Emerging Mindset of SWE Agent

Although individual trajectories can be interpreted through close reading, this approach does not scale to our 408-run corpus and does not yield observations that are comparable across models and experimental conditions. We therefore record each run using a standardized scheme R that preserves the turn structure and artifacts required for later projection, then treat the resulting traces as empirical material that becomes evidence only when analyzed through lenses. The lenses are ordered because different observations carry different levels of interpretive risk. Surface activity stays closest to the trace. Temporal shape, action transitions, semantic judgment, and repository footprint require progressively stronger interpretation. Because no single perspective is adequate for characterizing agent behavior, we organize the lenses hierarchically into four layers (Figure 3): the census layer, the narrative layer, the mechanism layer, and the evidence layer. The purpose of this layering is to support progressive reading: corpus-level summaries enable screening and comparison, while increasingly specific projections recover temporal structure, decision mechanisms, and repository-grounded evidence needed to substantiate explanatory claims. In the census layer, lens 0 (the surface activity view) summarizes each trajectory into activity statistics that establish a baseline for comparing models and conditions across the run matrix. It separates changes in overall effort from changes in higherlevel behavior. Its summary values recur as reference points for the other lenses. In the narrative layer, lens 1 (the journey arc lens) renders each trajectory on a normalized timeline so that many runs can be compared in a common visual frame. By foregrounding temporal structure instead of isolated events, it supports corpusscale interpretation through recurrent phases, loops, and atypical trajectories. In the mechanism layer, lens 2 (the action policy lens) characterizes tool use as sequential decision behavior by modeling adjacent tool transitions as n-gram patterns [10]. Table 5 defines the transition codes used in this work; the coding is apparatus-specific because it depends on Ada’s tool vocabulary and on the meaning of tool transitions in a code-understanding setting. In the mechanism layer, lens 3 (the automatic adjudication lens) adds semantic evaluation that cannot be recovered reliably from structural signals alone. Using LLM-based judging, it assesses whether a trajectory’s outcome is relevant to the task, grounded in repository evidence, appropriately calibrated at termination, and navigationally well-oriented; details and validation are reported in Appendix C. In the evidence layer, lens 4 (the codebase footprint lens) makes repository contact explicit by projecting each trajectory into a map of visited files and code regions (and their aggregate heatmaps). This view distinguishes focused evidence acquisition from diffuse or shallow exploration, grounding behavioral claims in where the agent actually interacted with the codebase. The lenses are intended to be read in combination rather than as stand-alone diagnostics. Lens 0 establishes a baseline of effort and tool use; lenses 1–2 recover temporal and sequential structure; lens 3 evaluates conclusion-level adequacy; and lens 4 grounds these readings in repository contact. Agreement across lenses strengthens interpretation; disagreements localize where a behavioral account is

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

15

Table 5: Action policy codes and readings. Action Policy

Operational Rule(s)

Interpretive Reading

close read

PKR → PKR$; EXP → PKR$

Sustained inspection of adjacent or continuous code regions.

skim

OVW → OVW; NAV PKR*; ANY → PKR*

→

Broad sampling across structures or file boundaries.

traverse

NAV → PKR; ANY → PKR

Ordinary movement from navigation into code reading.

trace

ANY → OVW*

→

Following code relations or shifting structural attention across files.

zoom in

OVW → PKR; OVW → EXP

Moving from structural overview toward local inspection or traversal.

zoom out

ANY → OVW

Returning from local action to structural overview.

resume

COG → NAV

Returning from cognitive work to repository exploration.

other read

ANY → NAV

Residual navigation transition not captured by more specific rules.

close out

NAV → CON terminal

Moving from repository navigation into final synthesis.

interim synth

NAV → non-terminal

CON

Recording a provisional conclusion without terminating.

dump

NAV → COG; ANY → COG

Externalizing notes, bookmarks, hints, or conclusions after prior action.

cognitive streak

COG → COG

Consecutive consolidation, deferral, clarification, or synthesis actions.

other

ANY → ANY

Fallback for transitions not matched by higher-precedence rules.

EXP;

ANY

Note: NAV = {OVW, PKR, EXP} and COG = {MEM, BKM, HNT, CON}. A * condition requires different current file targets. A $ condition requires continuous line ranges. Rules are applied by precedence: exact rules before wildcard rules; more specific wildcard classes before less specific ones; $ before *; * before regular wildcard rules; remaining ties by list order.

underdetermined, motivating closer inspection. The behavioral profiles in Section 4 are derived from this triangulation, and the set L remains extensible for additional questions and agent settings.

4 Results The results are organized to mirror the layered lens stack introduced in Section 3.4. We begin with the census layer (RQ1, Section 4.1), reporting surface signals that establish a baseline of effort and exposure. We then project trajectories through the narrative, mechanism, and evidence layers (RQ2, Section 4.2), moving progressively

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

16

from temporal shape through action transitions and semantic adjudication to repository footprints. Finally, RQ3 (Section 4.3) tests whether the intervention improves outcomes or exposes a boundary of the apparatus. Throughout, we report observer-oriented findings in a consistent three-part form: signals (what the lens makes measurable), profile (the comparative pattern the projection reveals), and boundary (where the projection ceases to support a stable reading).

4.1 RQ1: Surface Signals from the Apparatus Lens 0 (the surface activity view) projects each trajectory into a small set of censuslayer signals, namely turns, lines of code viewed, files visited, elapsed time, and per-turn effort, to support cross-model and cross-condition comparison (Figures 4– 5). At this level, models differ substantially in exploration scale under the natural condition, while budget pressure produces a consistent compression of surface activity across the run matrix.

10 0 s. ed ral ppr. Pres ombin Natu rior Su get C P Bud

1200 800 400 0 s. ed ral ppr. Pres ombin Natu rior Su get C P Bud GPT-4.1

GPT-4o

Avg. Time Elapsed (s)

20

1600

Avg. Files Viewed

30

Avg. LOC Viewed

Avg. Turns

40 12

8

4

0 s. ed ral ppr. Pres ombin Natu rior Su get C P Bud DeepSeek-V3.2

320 240 160 80 0 s. ed ral ppr. Pres ombin Natu rior Su get C P Bud

Qwen3.5-397B

Fig. 4: Surface-level exploration activity declines markedly under budget pressure and combined constraints across all four LLM backends. Each panel tracks averages. turns, LOC viewed, files viewed, and time elapsed (left to right) across four constraint conditions: natural, prior suppression, budget pressure, and combined.

Figure 4 makes the condition effects explicit. Under budget pressure, turns, LOC viewed, files visited, and total elapsed time decrease for all four models. Under the combined condition, models converge to similarly low activity levels, reducing the between-model spread visible in the natural condition. By contrast, the prior suppression condition yields surface profiles that remain close to the natural baseline across all four panels, indicating that this constraint is not distinguishable at the census layer.

Springer Nature 2021 LATEX template

Avg. Time per Turn (s)

16 12 8 4 0

d s. ral ppr. bine Pres r Su get Com Prio Bud

Natu

GPT-4.1

GPT-4o

Avg. Thinking Chars per Turn

Projecting the Emerging Mindset of SWE Agent

17

600

400

200

0

d s. ral ppr. bine Pres r Su get Com Prio Bud

Natu

DeepSeek-V3.2

Qwen3.5-397B

Fig. 5: Per-turn working effort intensifies under budget pressure, revealing a tradeoff between session breadth and turn-level depth. Two panels show the average time per turn (s) and the average thinking characters per turn across natural, prior suppression, budget pressure, and combined conditions for four models.

Budget pressure does not merely reduce the number of turns; it also changes perturn behavior. Figure 5 shows an inversion in the census-layer signals: aggregate activity decreases, while per-turn time and per-turn “thinking” characters increase. The increase is most pronounced for GPT-4.1 (time per turn rising from about 5 seconds to over 15 seconds), and is also visible for DeepSeek and Qwen, while GPT4o remains comparatively stable. Lens 0 makes this trade-off observable, but the trajectory record at this layer does not permit attributing the additional per-turn time to any specific source (e.g., model deliberation versus serving latency).

Finding 1 signals: Lens 0 projects each run into surface activity and effort measures (trajectory length, repository exposure, elapsed time, and per-turn effort). profile: Under the natural condition, models separate by exploration scale, whereas budget pressure produces a consistent compression of surface activity and a concurrent increase in per-turn effort. boundary: Prior suppression is not distinguishable at this layer, and surface signals do not determine whether observed time increases reflect agent deliberation or external latency.

4.2 RQ2: Projecting Trajectories through Observation Lenses Beyond the census-layer surface signals in RQ1, RQ2 projects trajectories through the narrative, mechanism, and evidence layers to make observer-readable structure explicit. Across these projections, models and constraints can exhibit markedly different temporal organization, transition grammars, semantic adequacy, and repository contact patterns, even when surface activity appears similar. We report these results in four views: journey arcs (Section 4.2.1), action policy transitions (Section 4.2.2), automatic adjudication (Section 4.2.3), and codebase footprints (Section 4.2.4).

4.2.1 Journey Arc: Temporal Shape and Within-Condition Variance The journey arc lens (lens 1) projects each trajectory into a normalized, observerreadable timeline, making temporal organization and within-condition variance

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

18 Markers

OVW BKM HNT

PKR

EXP

<20L 20-50L >50L

MEM

Def Ref Next

CON

High Medium Low

100% = 4.5 min (269044 ms)

Errors

Finalize Hold

Tool Error Unknown tool

100% = 4.2 min (249734 ms) natural condition | GPT-4o

natural condition prior suppression | GPT-4o

prior suppression

natural condition | GPT-4.1

scikit-learn

combined condition

prior suppression | GPT-4.1 scikit-learn

budget pressure

natural condition | DeepSeek-V3.2is

flask budget pressure

prior suppression | DeepSeek-V3.2

budget pressure

natural condition | Qwen3-235B

graphiti

tensorzero prior suppression | Qwen3-235B

budget pressure

0

25

50

Normalized elapsed time

75

(a) Journey arcs of GPT-4.1, under the task of risk and code-smell review, varying in conditions (arc 1–4) and repositories (arc 4–7).

100%

0

25

50

Normalized elapsed time

75

100%

(b) Journey arcs under the task of architecture analysis, varying all 4 models, pairing the natural and priorsuppression conditions.

Fig. 6: Journey arc lens examples. Each arc is a readable timeline with tool calls shown as markers, elapsed time running left to right, and the full journey visible as an overall contour.

directly comparable across runs. Figure 6 shows that constraints can change not only how much activity occurs, but also how that activity is distributed over time. Figure 6a traces GPT-4.1 trajectories for risk and code-smell review across conditions. Under budget pressure and the combined constraint, arcs are visibly shorter than under the natural condition, consistent with the census-layer compression in RQ1. In addition, the temporal projection reveals heterogeneous within-trajectory pacing: some budget-pressure runs contain extended gaps between successive actions without a commensurate increase in visible output. Figure 6b contrasts natural and prior suppression across models for architecture analysis. Within each model, the two conditions exhibit closely matched arc shapes, whereas the dominant differences are model-specific temporal signatures (e.g., more extended orientation-heavy arcs versus compact, direct arcs). The temporal gaps visible in some arcs raise an attribution boundary for observer-facing interpretation. For GPT-4.1, per-turn elapsed wall-clock time correlates only weakly with per-turn output length (Pearson r = 0.397; Spearman ρ = 0.532), so extended pauses in Figure 6a cannot be read unambiguously as additional agent work. We therefore treat unusually long per-turn intervals as a limitation of the projection under API-mediated execution; possible serving-stack contributors are discussed in Section 5.1.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

19

Figure 6b pairs natural condition and prior suppression trajectories for all four models on architecture analysis in scikit-learn. Within each model, the two conditions produce very similar journey shapes, making them hard to tell apart visually. The main contrast is model-level. GPT-4o and GPT-4.1 remain compact, DeepSeek spans the timeline with OVW-heavy orientation, and Qwen falls in between. At this level of reading, prior suppression leaves Ada’s outward journey largely unchanged. In other words, the condition appears to alter evidential stance without reshaping the visible arc of exploration.

Finding 2 signals: Lens 1 projects each trajectory as a time-ordered arc, preserving within-run sequencing and per-turn timing. profile: The projection makes withincondition variance visible. Budget pressure yields shorter arcs and, in some runs, uneven pacing with long inter-action gaps, while natural and prior suppression arcs within a model are visually similar relative to the larger model-to-model differences. boundary: Wall-clock timing in the arcs is not uniquely attributable to agent work; extended pauses should be treated as an execution artifact boundary under API-mediated serving.

4.2.2 Action Policy: Transition Signatures across Models Across all 384 trajectories, raw tool-call counts are highly concentrated. peek code region and get structural overview together account for 80.1% of all tool calls, and this concentration holds across models and conditions (Appendix D). Lens 2 (the action policy lens), therefore, shifts the reading from marginal tool frequencies to sequential structure by coding adjacent tool-to-tool transitions (Table 5). Figure 7 shows that these transition profiles are modelcharacteristic, and that budget pressure reshapes them in distinct, model-specific ways rather than producing a single uniform “compressed” policy. Figure 7 reports transition-tag shares by model and condition. It supports two observer-facing readings: models exhibit stable transition signatures, and constraints reshape these signatures in model-specific ways.7 Model-wise signatures. In Figure 7, GPT-4o emphasizes close reading; DeepSeek is traverse-heavy in line with its OVW-style orientation sequences; Qwen exhibits the highest skimming share; and GPT-4.1 is comparatively evenly distributed across tags. Condition-wise shifts. Natural and prior suppression remain closely matched within each model. Budget pressure changes the transition mix, but without a single common direction: GPT-4.1 shifts toward cognitive transitions and away from close reading; GPT-4o reduces close reading while increasing dump and zoom-in; DeepSeek becomes more evenly distributed; and Qwen trades skim for zoom-in.8 Task family also modulates these transition profiles in consistent directions, and the task-stratified results are reported in Appendix D. 7 The combined condition is omitted from this figure for compactness. Its profiles closely resemble budget pressure across all models and are reported in Appendix D. 8 Close-out share rises mechanically under budget pressure because shorter trajectories contain the same single termination action against fewer total transitions. This is a denominator effect, not a behavioral shift.

Springer Nature 2021 LATEX template 20

Projecting the Emerging Mindset of SWE Agent GPT-4.1 Natural 22

14

20

14

Prior Suppression 20

15

20

17

Budget Pressure 13

12

15

16

GPT-4o Natural 27

13

16

15

Prior Suppression 31

15

17

12

Budget Pressure 14

17

18

18

DeepSeek-V3.2 Natural 13

24

19

18

Prior Suppression 12

27

21

17

Budget Pressure 20

20

20

19

18

12

Qwen3.5-397B Natural 17

26

Prior Suppression 13

24

22

23

Budget Pressure 17

0

15

20

25

21

50

15

75

100

Share of action policy tags (%) close read traverse trace

zoom in skim zoom out

dump resume cognitive streak

interim synth close out

Fig. 7: Action policy lens: transition-tag shares across models and conditions. Each row summarizes one model under natural, prior suppression, and budget pressure. Stacked bars show the percentage composition of adjacent tool-to-tool transitions, grouped into the transition tags defined in Table 5 (e.g., close read, traverse, skim, zoom, dump, cognitive streak, close out, and intent switch).

Finding 3 signals: Lens 2 projects trajectories into transition-tag shares over adjacent tool-to-tool moves (Table 5; Figure 7). profile: The resulting transition signatures are model-characteristic, vary systematically with task family, and remain similar between natural and prior suppression. Budget pressure changes the transition mix, but the direction of change depends on the model rather than following a single uniform compression pattern. boundary: Transition profiles describe sequential structure but do not evaluate whether a given action policy produces higher-quality conclusions; this requires semantic adjudication in lens 3.

Models navigate differently and reorganize differently under pressure. Whether these different action policies produce different quality outcomes is a question the action policy lens cannot answer. This is the opening for the adjudication lens.

4.2.3 Adjudication: Conclusion Quality and Epistemic Grounding To evaluate whether different projected policies yield different outcomes, lens 3 (adjudication) applies LLM-based judges to each trajectory conclusion on five

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

Task-relevance score (normalised)

1.00

21

Model GPT-4.1 GPT-4o DeepSeek-V3.2 Qwen3.5-397B

0.75

Condition Natural Prior Suppr. Budget Press. Combined

0.50

0.25

0.00 0.6

0.7

0.8

0.9

1.0

Conclusion-grounding score (normalised, truncated at 0.6)

Fig. 8: Automatic adjudication scores for task relevance and conclusion grounding, normalized to [0, 1]. Points are aggregated by model and condition, showing how conclusion quality varies across the run matrix. Color encodes the model, and marker encodes the condition.

Termination calibration (n=383)

Navigational orientation (n=6309)

Synthesis orientation (n=672)

1.00

Normalised score

0.75

0.50

0.25

0.00 GPT-4.1

GPT-4o

DS-V3.2 Qwen-3.5 Natural

GPT-4.1

GPT-4o

Prior Suppr.

DS-V3.2 Qwen-3.5 Budget Press.

GPT-4.1

GPT-4o

DS-V3.2 Qwen-3.5

Combined

Fig. 9: Automatic adjudication scores for termination calibration, navigational orientation, and synthesis orientation, shown left to right. Boxplots summarize per-trajectory distributions by model and condition, and points show individual trajectories. The figure highlights how these dimensions shift under different constraints.

dimensions: task relevance, conclusion grounding, termination calibration, navigational orientation, and synthesis orientation. Figures 8–9 show that adjudicated quality is primarily model-characteristic, while constraints produce more localized shifts (most clearly in termination calibration under pressure).

Springer Nature 2021 LATEX template 22

Projecting the Emerging Mindset of SWE Agent

Table 6: Automatic adjudication scores for navigation grounding and synthesis grounding. The table reports task-by-model means, with each cell showing navigation grounding/synthesis grounding. “Overall” is the task-level mean across models. Task

Overall

GPT-4.1

GPT-4o

DeepSeek-V3.2

Qwen3.5-397B

architecture analysis

0.691/0.558

0.666/0.481

0.751/0.804

0.674/0.400

0.681/0.375

ASR inference

0.615/0.410

0.568/0.367

0.698/0.631

0.600/0.268

0.607/0.370

feature implementation tracing

0.712/0.588

0.695/0.604

0.736/0.646

0.715/0.614

0.711/0.389

risk and code smell review

0.685/0.570

0.691/0.516

0.730/0.644

0.658/0.658

0.681/0.333

Figure 8 summarizes conclusion-level quality by plotting task relevance against conclusion grounding for each model-condition pair. The primary pattern is modelwise clustering: T1 ranges from 0.174 (GPT-4o) to 0.868 (Qwen), while conditionlevel variation remains within 0.608–0.632. A closer reading highlights GPT-4o as an outlier: despite high trace grounding (T2 = 0.845) and strong close-reading share in lens 2, its task relevance is substantially lower than the other models. A constraint can change how the agent writes, qualifies, or justifies its conclusions more easily than it changes where the agent navigates. This creates a possible mismatch between evidential stance and evidence-gathering behavior. Figure 9 reports termination calibration, navigation grounding, and synthesis grounding by model and condition. Here, the main condition-level shift is in termination calibration, which decreases under pressure (0.715 under natural versus 0.633 under budget pressure). Navigation grounding and synthesis grounding change more subtly across conditions. Under prior suppression, synthesis grounding decreases slightly, while navigation grounding remains similar or shifts marginally in the opposite direction. At the action level, structural overview actions show the strongest prior-leaning navigation, with a grounding score of 0.464 compared to 0.827 for explore-code actions. Conclusion and memory actions show the strongest prior-leaning synthesis, with scores of 0.497 and 0.525. This asymmetry is consistent with prior suppression affecting synthesis-oriented output more strongly than navigation-oriented behavior. We revisit possible interpretations of this mismatch in Section 5.1. Table 6 reports grounding scores by task family. ASR inference is the most priorshaped task on both dimensions (navigation: 0.615, synthesis: 0.410), consistent with a task that requires speculating beyond what static code reveals.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

23

Finding 4 signals: Lens 3 projects trajectory outcomes into adjudicated conclusionlevel and epistemic-grounding scores (Figures 8–9; Table 6). profile: Adjudicated quality is primarily model-characteristic and comparatively stable across conditions. The projection also reveals mismatches between dimensions, including cases where high grounding co-occurs with low task relevance (Figure 8). Under budget pressure, termination calibration decreases, while navigation and synthesis grounding shift more subtly (Figure 9). boundary: These scores identify where trajectories align or misalign with the judged criteria, but they do not by themselves establish a causal mechanism for the observed mismatches; interpretation is deferred to Discussion (Section 5).

4.2.4 Footprint: Spatial Divergence in the Repository World Lens 4 (the codebase footprint lens) projects each trajectory into a spatial tracedensity map over files and code regions, making repository contact directly comparable across runs. This evidence-layer projection reveals systematic differences in spatial attention across models, constraints, and tasks, while also preserving meaningful single-run variation. Figure 10 visualizes these footprint projections. Panel (a) summarizes aggregate footprints across all six repositories and shows that large regions remain unvisited, with attention concentrated on a subset of structurally central files. Panel (b) contrasts models on the same repository and task, showing that spatial attention can diverge markedly even when the experimental configuration is held fixed. Panel (c) contrasts conditions and shows spatial contraction under budget pressure, with coverage narrowing to fewer files, while retained hotspots can intensify locally. Panel (d) contrasts tasks and shows that architecture analysis induces broader coverage than feature implementation tracing, which concentrates on a smaller module neighborhood. In addition to these aggregate comparisons, the footprint projection remains informative at the run level. Figure 11 shows that, even for the same model, task, and repository, individual trajectories can follow different spatial routes through the codebase. A boundary of the footprint projection is that it inherits artifacts from the underlying tool interface. For example, some configurations exhibit anomalous single-step intake (e.g., reads exceeding 1,000 LOC in one action), which can inflate local density independent of a sustained exploration strategy.

Finding 5 signals: Lens 4 projects repository contact as trace-density over files and code regions. profile: The footprint projection distinguishes spatial attention patterns by model, shows coverage contraction under budget pressure with locally intensified hotspots, and reflects task-dependent coverage differences. It also preserves single-run variability, with different trajectories traversing different regions under the same configuration. boundary: Density is shaped by the repository interface (e.g., occasional single-step large-LOC reads), so local hotspots should be interpreted as evidence of contact rather than as a direct measure of sustained effort.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

24

pt-optimize prom r

(a) overall footprints

Trace Density (LOC)

packages

csorrce

0 LOC

services

700+ LOC

e

mplat te

− ui

i18nsrc

scikit-learn

web

sklearn tensio

e

svm

ar r a

eq

r

b en

pecti ins _plot on ma

flca/flsaks

nifold

y

k

sr

ctiiton s

f

n ibli ea

uest

servabil un

_scipy

s

ataset

s

d

ev

ns

ob

(e) demonstration of signed density

r

aluatio

ui

r_mo

_lib

routes

ponen ts

rust

metrics

ea

l

at

com

mp

clients

graphiti

_plot

tree

hiti_co r rap

e

ug tils

ghbors ne i

p o m os

e

c

v ser er

utils

odel

s

m

mi

xture

README.md LICENSE graphiti_mcp_serve...

_r

r_h ep

cluster

tml

README.md

c hm a

de

api_co

dya_sk

utils

li n

ce

externals

app

a alusrctio

h p y t on

tree

rk s

atasets

ddata

ui

feren

ns

ev

rian t

va

m nse bl

utils

l

s

in

vider

e

e

c

e

pro

sorzser ro-cor

cluster

src

tensorzero

ten

=

n

pxublic

u imp te c

estimato...

ov

arian

hd

bsc

search utils

(d) task-wise comparison

an

_

graphiti_core

sansio

search_f...

maintenance

search_utils.py

search.py

app.py

scaffold.py

prompts search_co...

edge_operatio...node_operati...

eval.py

sessions.py blueprints.py

templating.py

community_...

extract_no...

debughelpers.py

nodes.py

models g e d es

typing.py blueprints.py

ctx.py

llm_client

bedder em

app.py

client.py openai_g...

Signed Trace Density (LOC) -700 LOC

logging.py

edges.py d es

testing.py

extract_e...

graphiti.py

wrappers.py

0

700+ LOC

Bubble Styles cli.py

size = LOC

(b) model comparison

(c) condition-wise comparison

Fig. 10: Repository footprints projected through trace density. Panel (a) shows an overview of all six repositories with cumulative trace density across the full matrix. Panel (b) compares models: signed density map of Qwen3.5-397B minus DeepSeekV3.2 on Graphiti, showing divergent spatial attention on the same repository and task. Panel (c) compares conditions: signed density map of Natural minus budget pressure on Flask, showing spatial contraction under constraint with locally intensified heat zones. Panel (d) compares tasks: architecture analysis minus feature implementation tracing on scikit-learn, showing broad architectural exploration against narrow implementation focus. Panel (e) provides the legend for signed density encoding.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

25

File Types

Bubble Styles

paacckkaaggeess p

.js .json .ts .ui .yaml

cco orre e

ssrrcc

size = LOC

sse errvviicceess

orraaggee sstto

striped = viewed structure

4

Line Style dexieStorageProvid... dexieStorageProvid...localStorag... localStorag... 3 5 mpptt prroom p

4

service.ts service.ts

service.ts service.ts

history history mppllaattee em tte

mooddeell m

2ddeeffaauulltt--tteem mppllaatt miizzee oppttiim o

eess

analytical... analytical...

manager.ts manager.ts

4

2 3

natural prior suppression budget pressure m llllm

2

user-opti... user-opti...

6 5

1

manager.ts manager.ts

1 3

static-loade... static-loade... types.ts types.ts 1

pnpm-lock.yaml pnpm-lock.yaml

ui ui

extension extension

web web

Fig. 11: Under different conditions, even the similar prior suppression condition and natural condition are spatially different. They might have overlaps on files, but the order is unpredictable. Budget compression shows a more compressed amount of navigation in this view.

Summary of RQ2. The four observation lenses, read together, project a consistent pattern from different vantage points. Models carry distinct behavioral profiles that are visible in their journey shapes (lens 1), action grammars (lens 2), quality signatures (lens 3), and spatial footprints (lens 4). Budget pressure compresses and restructures these profiles in model-specific directions, but does not collapse conclusion quality. Prior suppression is invisible to the surface, temporal, and action-level lenses, and becomes detectable only through the adjudication lens, where it constrains synthesis more than navigation. Across all lenses, the same finding recurs at increasing resolution: trajectories diverge while conclusions converge. Different models, different conditions, and even different runs of the same configuration produce distinct journeys through the repository world, yet arrive at comparable understanding. The projection protocol makes this divergence visible and analyzable. What it cannot do is intervene. Whether an external intelligence can improve the trajectory rather than merely observe it is the question RQ3 addresses.

4.3 RQ3: Does Intervention Improve or Expose a Boundary? RQ3 evaluates an intervention condition in which a judge panel and a coach are introduced into GPT-4.1’s observation channel, yielding 24 additional trajectories across the same repositories and task families. We read this condition through the same lens stack as RQ2 (Figure 12, plus the footprint projection in Figure 13). At the level of these projections, coaching increases trajectory volume but is associated with a shift toward late-phase conclusion cycling and reduced conclusion grounding. Relative to the natural condition, the coach condition generates substantially more surface activity: 2.5× more turns, 3.4× more files visited, and 3.2× more total

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

26 Markers

PKR

EXP

MEM

CON

Errors

Coach

Natural condition 23

15

21

14

Coach condition if naturally ended 20 tensorzero Natural Condition | GPT-4.1 | Architecture Analysis

16

20

16

Coach condition extended lifetime 23

16

23

21

Coach condition overall 23

tensorzero Natural Condition | GPT-4.1 | ASR Inference

0

16

25

17

50

14

75

100

Share of action policy tags (%) tensorzero Coach | GPT-4.1 | Architecture Analysis

tensorzero Coach | GPT-4.1 | ASR Inference

(a) lens 1 - journey arc

close read traverse trace

zoom in skim zoom out

dump resume cognitive streak

interim synth close out

(b) lens 2 - action policy

(c) lens 3 - automated adjudication

Fig. 12: Coach-judge condition observed through three lenses, all on GPT-4.1. (a) Journey arcs comparing natural condition and Coach Condition on TensorZero across four task families. Natural trajectories terminate within the first quarter of the coach timeline. Coach trajectories extend substantially, with late-phase segments dominated by repeated attempts to conclude and coach rejections. (b) Action policy signatures split by lifetime regime. The natural-lifetime portion of coach trajectories closely resembles the natural condition. The extended-lifetime portion shifts toward close-out and cognitive-streak dominance, indicating a regime change from exploration to conclusion cycling. (c) Adjudication scores comparing Natural and Coach conditions. Conclusion grounding drops under coaching; Task relevance spreads wider with more extreme low scores; Termination calibration remains stable.

time. Per-turn time remains nearly identical (5.50 s vs. 5.32 s), so the increase is primarily in trajectory length rather than per-turn duration. The amount of extension also varies by repository; some trajectories terminate earlier, while others (notably on TensorZero) extend to the 100-turn bound. Inside Figure 12, panel (a) shows that natural-condition trajectories on TensorZero terminate within the first quarter of the coach-condition timeline, whereas coach-condition trajectories extend across the full duration and exhibit late-phase segments dominated by repeated conclusion attempts and coach rejections. Panel (b) shows the corresponding transition-profile split: the natural-lifetime portion of coach trajectories remains close to the natural condition, while the extendedlifetime portion shifts toward close-out (21% of transitions) and cognitive streaks, with reduced close reading and traversal. Panel (c) reports adjudication outcomes. Under coaching, conclusion grounding shifts from a tight natural-condition range around 0.85–0.95 to a lower median

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

27

tteennssoorrzzeerroo--ccoorree ssrrcc prroovviiddeerrss p

Repository tensorzero Bubble Styles

ooppeennaaii

size = LOC

mod.rs mod.rs together.rs together.rs google_ai_s... google_ai_s... deepseek.rs deepseek.rs

tgi.rs tgi.rs

sglang.rs sglang.rs gcp_vertex_gem... gcp_vertex_gem... anthropic.rs anthropic.rs

aws_bedr... aws_bedr...

mistral.rs mistral.rs gcp_vertex_... gcp_vertex_... fireworks.rs fireworks.rs

error.rs error.rs

e

ro

or

e

attiioonn ua allu vva

chat_comple...mod.rs chat_comple... mod.rs chat_comple... mod.rs

function.rs function.rs

s

config_parse... model.rs config_parse... model.rs

s

uuttiillss

orrzzee nnsso

et

te

apppp a

vvaarriiaanntt

mod.rs mod.rs

eennddppooiinnttss oaauuppttiieess rro

inference.rs batch_inf... inference.rs batch_inf...

Signed Trace Density (LOC) -150 LOC

0

150+ LOC

Fig. 13: Signed trace density map comparing coach condition minus natural condition for GPT-4.1 on TensorZero. Orange and red zones indicate files where the coach condition reads more code; blue zones indicate files where the natural condition reads more. The map visualizes how spatial attention shifts under intervention, including dispersion across a broader set of files versus concentration on fewer files with deeper coverage.

(around 0.68) with wider spread, and task relevance becomes more dispersed with more extreme low scores. Termination calibration remains similar between conditions, indicating that extended trajectories do not yield improved calibration signals under this projection. Figure 13 provides the footprint projection for the coach condition relative to the natural condition on TensorZero, making the intervention’s spatial redistribution of repository attention visible.

Finding 6 signals: Under intervention, lens 1 exposes extended trajectory lifetime and late-phase segments dominated by repeated conclusion attempts and coach rejections (Figure 12a); lens 2 splits transition profiles by regime and shows a shift in the extended lifetime toward close-out and cognitive-streak transitions (Figure 12b); lens 3 reports adjudication changes in grounding and relevance (Figure 12c); and lens 4 maps the corresponding spatial redistribution of repository attention (Figure 13). profile: Coaching increases trajectory volume and extends runs, but the additional lifetime is associated with a transition-profile shift away from exploration and with lower, more variable conclusion grounding, while termination calibration remains similar between conditions. boundary: In this apparatus, the intervention channel increases activity without yielding improved grounding signals under adjudication. This constitutes an

Springer Nature 2021 LATEX template 28

Projecting the Emerging Mindset of SWE Agent empirical boundary for externally steered trajectories in our setting: additional interaction does not necessarily translate into stronger repository-grounded conclusions.

5 Discussion 5.1 Situating Our Findings in the Broader Literature Ada’s apparatus was designed for behavioral characterization rather than largescale causal inference. Its findings are structurally reliable, grounded in 384 trajectories across four models, six repositories, and four constraint conditions, but they are best interpreted alongside related work. Prior studies address adjacent phenomena, including agent efficiency, diversity of reasoning paths, chain-of-thought faithfulness, epistemic constraints, and multi-agent coordination failures. Table 7 situates Ada’s findings in this landscape by pairing each observation with the closest evidence in the literature and its citation. In several cases, Ada recovers a known effect at trajectory resolution; in others, it provides process-level characterization that outcome-centric studies do not expose. Table 7: Ada’s observed patterns aligned with prior work. Each row pairs one observation from Ada’s projection study with the closest evidence in the community literature and its citation. Observed in Ada

Closest findings in prior works

Citation

Different models traverse different repository regions yet arrive at comparable conclusions

Complex reasoning tasks admit multiple diverse reasoning paths that reach the same correct answer

[11]

Different SWE agents resolve very different sets of issues despite having similar resolve rates; the large action space of agents inevitably produces solution diversity

[12]

Agentic system evaluation has been mostly outcome-centric, overlooking intermediate steps and masking recurrent inefficiencies; process-centric metrics are needed to assess quality independent of final success

[1]

LLM-generated reasoning chains are often not causally linked to the final answer; perturbing intermediate steps may not change predictions, making explanations essentially post-hoc

[13]

Longer reasoning / extra turns show diminishing returns: stopping earlier can maintain comparable accuracy, and max-turn trajectories are unlikely to yield substantial gains.

[14, 15]

Inefficiency and budget interaction: redundant steps waste compute without improving outcomes, and budget effectiveness depends on scaffold–LLM synergy (with failed attempts consuming disproportionate tokens).

[16, 17]

Conclusion quality is modelcharacteristic and largely condition-stable; the outward surface of behavior is a weak predictor of synthesis quality

Budget pressure compresses trajectory volume while conclusion quality remains largely stable, suggesting unrealized efficiency in unconstrained trajectories

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent Observed in Ada

Closest findings in prior works

Citation

Wall-clock time gaps in API-mediated trajectories create an attribution boundary for temporal projections; pauses cannot be interpreted uniquely as additional agent work

Latency variability in LLM serving can confound time-based measures, and gray-failure dynamics can manifest as partial slowdowns without fail-stop errors

[18, 19]

Prior suppression constrains synthesis more than navigation; agents adjust written claims more than evidencegathering behavior, consistent with epistemic specification gaming

Specification gaming occurs when agents satisfy the literal objective but not its intent; LLMs generalize from simple forms of gaming to more sophisticated compliance failures that are nontrivial to remove

[20]

LMs often ignore provided context when it conflicts with preexisting parametric memory; navigational behavior draws on ingrained priors that prompt-level instructions cannot easily override

[21]

LLM teams consistently underperform their best individual member by 8–38%; the failure is not identification but leveraging, because teams average expert and non-expert views rather than appropriately weighting expertise

[22]

Multi-agent integration success degrades monotonically as shared specification decreases; a substantial coordination tax persists even with full specifications

[23]

Diversity only helps with good aggregation: synthesis can lose to single-model baselines, while selection/competition can beat synthesis

[24]

Role-specialized pipelines can silently propagate errors without auditable handoffs; critique is not independently verified before reuse

[25]

External coach override reduces trace grounding while leaving natural termination calibration intact; added interaction increases activity but degrades outcomes

Coach fuses multi-judge critiques into a single instruction; this lossy aggregation can discard the strongest signal and obscure responsibility boundaries

29

Across the seven rows, we see a consistent pattern. Our experiments are small in scale, but the composed methods and lenses let us project trajectories from multiple views. The mechanisms we observe align with prior work at a larger scale, including inefficient exploration, divergent reasoning paths with similar endpoints, output shifts that exceed behavior change, and coordination loss in multi-agent systems. Ada adds structure by making these effects visible in the layers of narrative, mechanism, and evidence. It also pinpoints specification gaming at the navigationto-synthesis boundary and shows how oversight can push agents from grounded exploration into conclusion cycling. The projection protocol is what makes these dynamics visible.

5.2 Making the Code-Understanding Journey Observable The contribution of this work is observational. By treating trajectories as empirical objects, the apparatus and lenses make repository-level code understanding comparable as a process: what evidence is inspected, how navigation and revisitation unfold, when intermediate understanding is externalized, and how stopping is calibrated. These comparisons identify projected behavioral differences under controlled launch conditions. They do not by themselves establish causal mechanisms inside the model.

Springer Nature 2021 LATEX template 30

Projecting the Emerging Mindset of SWE Agent

This framing also bounds the interpretation of the launching conditions. Differences across conditions should be read as behavior under controlled changes in guidance within a fixed instrument, not as intrinsic properties of the underlying models or as guarantees of transfer to other software environments. More broadly, Ada provides a reusable observation design for SWE agents. It projects externalized behavior through explicit lenses to support responsible, process-level claims without assuming direct access to internal cognition.

5.3 What Can Travel Beyond Ada The preceding subsections situate Ada’s findings in the literature and clarify the observational contribution of the apparatus and lenses. What can travel beyond Ada is therefore not the specific behavioral profiles reported here, but the observation logic that makes such profiles readable. The method is transferable only with design work. Other researchers should not treat Ada as a universal template that can simply be copied across agents, tools, or domains. What can travel is the logic of the apparatus. A future study can define a bounded software world, specify meaningful task families, expose tools that shape the agent’s contact with the environment, and design lenses that match the phenomenon under observation. Under that condition, trajectory-centered projection can become a practical method for studying SWE agents across different contexts. The findings here are not the end of that program. They are an argument that such a program is possible.

6 Threats and Limitations The limitations of this study are part of the methodological conditions under which trajectory-level observation becomes possible. Ada was designed to project repository-level code understanding from behavioral traces, while treating the underlying models as black boxes. This gives access to how an agent moves, reads, records, revises, and terminates inside a bounded repository world. It does not give access to why the model internally preferred one hypothesis, file, or conclusion over another. The three limitations below clarify the epistemic boundaries within which the reported findings hold.

6.1 Apparatus Dependence Any attempt to study an SWE agent in action requires deciding what world the agent enters, what actions it can take, what feedback it receives, and what counts as completion. These decisions constitute the observation environment, and they shape the trajectories that result. Ada is therefore not a neutral window onto SWE agent behavior. Its task openings, prompt scaffold, tool surface, repository interface, and termination mechanism all define what becomes observable, and what remains outside view. The four experimental conditions are prompt-level and interaction-policy perturbations over a single scaffold. They do not change the underlying toolset or runtime architecture, which strengthens cross-condition comparison but means that

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

31

observed behavioral differences depend on how the perturbations were designed. Model selection introduces a parallel dependence. The four foundation models used for trajectory generation, the post-hoc evaluation models, and the runtime coachjudge layer belong to different parts of the experimental stack, and the study cannot claim that the same profiles would appear under all model families or all prompt formulations. Our design addresses this dependence by keeping the core scaffold stable across conditions and changing only specific forms of guidance or decision pressure. The resulting trajectories should be read as behavior under Ada’s apparatus. They speak to this setting, and the apparatus is designed for structured observation across conditions, not for claims about SWE agents in the abstract. Precisely because this dependence is unavoidable, making it explicit is a methodological contribution. The community benefits from observation environments whose assumptions are inspectable, rather than from claims that appear apparatus-free.

6.2 Scope of Repository-Level Code Understanding Code understanding is one phase of software engineering work, and this study deliberately isolates it. The task families ask the agent to understand, trace, infer, and explain code. They do not ask it to patch bugs, run tests, manage issue histories, negotiate requirements, or maintain software over time. This scope removes patch success as the organizing endpoint and places code understanding itself at the center of observation. It also means that the findings should not be transferred directly to repair-centered SWE agent settings. The repository sets bounds on the study in a second way. Six repositories span compact libraries, classic frameworks, scientific software, and recent AIinfrastructure systems, providing a structurally heterogeneous terrain for observation. Heterogeneity, however, is not coverage. Six repositories cannot represent the full space of software systems, organizational practices, programming languages, or development contexts. The scale difference across the set, from compact repositories to systems exceeding 100K LOC, helps stress the apparatus but also means that repository complexity may interact with model behavior and task type in ways this study can only begin to characterize. These boundaries are intentional. They define the experimental field within which the observation method can be demonstrated and assessed, while leaving the question of broader applicability to future work with expanded repository sets and task designs.

6.3 Lens Dependence Behavioral profiles are projections, not the agent’s raw properties. Each lens makes certain aspects of the trajectory visible while leaving others outside the view. Surface statistics, tool-use sequences, grounding judgments, prior-usage assessments, and termination evaluations each project a partial reading. Different lenses could emphasize different phenomena, such as uncertainty management, hypothesis revision, semantic compression, or interaction with repository topology. This partiality

Springer Nature 2021 LATEX template 32

Projecting the Emerging Mindset of SWE Agent

is inherent to the method, and it carries specific epistemic consequences that the reported findings cannot fully resolve. One consequence concerns convergence. When trajectories diverge across models, but conclusions converge, the observation lenses cannot definitively distinguish genuine understanding from training-data recall or from task conditions that admit only a narrow range of acceptable answers. Prior suppression changes the evidential stance of the launch, but it operates at the prompt level and cannot guarantee the absence of parametric familiarity. Honest uncertainty about what convergence means is part of the current result. A second consequence concerns the automatic adjudication lens specifically. This lens uses LLM-based judges to evaluate LLM-produced trajectories. The design mitigates circularity through model separation, anonymization, and multi-model voting, but the structural condition remains. Judges and agents share overlapping training-data distributions, so agreement among judges may reflect shared priors rather than independent validation. The alignment between sampled human evaluation and judge scores provides partial reassurance, but it does not eliminate the concern. These limitations point toward the same methodological reminder. A trajectory does not interpret itself. Responsible study of SWE agent behavior requires explicit lenses whose assumptions can be inspected, reused, criticized, and redesigned. The set L demonstrated in this paper is open and minimal. Its value lies in showing that structured observation yields distinguishable behavioral signals, and in making the instruments available for the community to extend and refine.

7 Related Works 7.1 Trajectory-Aware Evaluation of Software and Tool-Using Agents Existing work on software agents has increasingly moved beyond final-answer evaluation toward process-aware analysis. Early evaluations of coding agents often centered on whether a task was completed, a patch passed tests, or a benchmark item was solved. More recent work treats the intermediate trajectory as an empirical object in its own right. Studies of thought-action-result logs, diagnostic toolkits for agent trajectories, and trajectory-level evaluation frameworks examine how agents search, read, call tools, recover from errors, and terminate across multi-step processes [3, 10, 26]. This shift is important because long-horizon agent behavior cannot be adequately understood from the final output alone. A correct result may be reached through inefficient, fragile, or poorly grounded routes, while an incorrect result may still contain informative evidence about where the agent’s process broke down. In this sense, trajectory-level analysis has become a necessary complement to outcome-level evaluation.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

33

7.2 Tool-Mediated Interaction and Repository Environments Within software engineering, this process-aware turn has been especially visible in work on repository-level agents. Recent benchmarks and datasets place agents inside realistic code environments, where success depends on locating relevant files, interpreting dependencies, maintaining state across turns, and handling setup or validation constraints [27–30]. These studies make clear that software agents do not operate over isolated snippets. They work inside repository worlds composed of files, symbols, dependency relations, build structures, tests, and developer tools. The resulting behavior is therefore not only a property of the model. It is co-produced by the model, the task opening, the action surface, and the feedback returned by the environment. A parallel line of agent research has studied tool-mediated interaction more generally. Tool-use benchmarks and surveys emphasize that agents must select actions, interpret stateful feedback, manage tool dependencies, and adapt over multiple turns rather than merely produce a single response [31–33]. This literature supports a central assumption of our apparatus. The tool interface is not a neutral channel through which an agent simply accesses information. It shapes what can be observed, what can be attempted, and what can become part of the trajectory. For SWE agents, this point is especially consequential because code-reading tools, structural overviews, reference tracing, memory mechanisms, and conclusion actions define the agent’s mode of contact with the repository. A trajectory should therefore be read as a tool-mediated encounter between an agent and a software world.

7.3 Repository-Level Code Understanding and Software Comprehension Repository-level code understanding provides the second foundation for this work. Modern LLM-based code research has moved from single-function or single-file tasks toward cross-file completion, repository-level editing, dependency understanding, context retrieval, and code question answering [34–38]. These works show that repository understanding is not merely a matter of giving the model a larger context window. It requires selecting relevant evidence, following dependencies across files, distinguishing central from peripheral code, and forming a coherent account of how parts of a system interact. This view is close to the problem setting of our study, but our focus differs. We do not ask whether an agent can complete, edit, or answer a repository task as an endpoint. We ask how its code-understanding journey can be made observable while it moves through the repository. The task families used in our apparatus also connect to long-standing software engineering research. Architecture analysis and architecture reconstruction have studied how high-level design structure can be inferred from implementation artifacts [39, 40]. Work on architecture evaluation and architecturally significant requirements explains why some design decisions and implicit requirements have disproportionate structural impact [41, 42]. Feature location research studies how

Springer Nature 2021 LATEX template 34

Projecting the Emerging Mindset of SWE Agent

user-visible behavior can be traced to source-code units, while inspection and codereview research grounds risk-oriented reading in established maintenance practices [43–45]. These traditions matter because they prevent repository-level code understanding from being treated as a new capability invented by LLMs. Architecture reasoning, requirement inference, feature tracing, and risk review are established forms of software comprehension. Our work brings these forms into an apparatus for agent trajectories.

7.4 From Process Evaluation to Projected Observation Taken together, these bodies of work establish three premises. First, final outcomes alone are too narrow for understanding long-horizon agent behavior. Second, agent trajectories are shaped by the tools and environments through which agents act. Third, repository-level code understanding is a cross-file, evidence-selective, and historically grounded software comprehension problem. However, existing work still leaves a methodological gap. Trajectory-aware studies often use process traces to diagnose failure, improve repair performance, train agents, or benchmark execution. Repository-level code benchmarks often retain task success as the organizing endpoint. Classical software comprehension explains what kinds of understanding matter, but it does not address how an LLM-based SWE agent’s understanding process can be observed from outside. This paper occupies that gap. We construct Ada as a scoped SWE agent apparatus for observing repository-level code-understanding trajectories in a wildbut-bounded software world. Ada is not intended as a benchmark agent, and its trajectories are not treated as transparent access to internal cognition. Instead, each trajectory is recorded as a recoverable observation record and projected through analysis lenses that make behavioral and epistemic profiles discussable. This positioning distinguishes our work from outcome-centered SWE agent evaluation and from repair-oriented trajectory diagnosis. Prior work shows that agent trajectories useful for evaluation, diagnosis, correction, and scaling. We use trajectories to make repository-level code-understanding journeys observable.

8 Conclusions This paper introduced Ada, a scoped SWE agent apparatus for observing repositorylevel code-understanding as a trajectory-level empirical object. By combining a bounded-yet-wild repository world, a read-only tool-mediated interface, and a reusable suite of observation lenses, the method makes agent exploration, synthesis, and stopping behavior comparable across models and controlled perturbations without treating internal cognition as directly accessible. Across 408 recorded trajectories, our projection study surfaced several stable signals. Budget pressure consistently compressed activity volume while leaving conclusion quality largely intact, revealing unrealized efficiency in unconstrained trajectories. Prior suppression altered the stance of synthesis more than the shape of navigation, suggesting that prompt-level evidential constraints are easier for agents to satisfy in their written claims than in their evidence-gathering behavior.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

35

Finally, injecting an external coach-judge sidecar increased interaction and activity but could degrade grounding, highlighting a coordination tax at the boundary where a solo trajectory becomes a multi-intelligence process. Taken together, these results argue for a methodological shift: progress on SWE agents should be evaluated not only by endpoints, but by projected observation of the process that produces them. Ada is not a universal template; it is an observation logic that can be redesigned for other domains, tool surfaces, and task families. Future work should expand the repository set and task openings, add lenses for uncertainty management and hypothesis revision, strengthen validation via human and non-overlapping automated judges, and connect trajectory-level signals to downstream engineering outcomes (e.g., repair success) while preserving the interpretability that trajectory-centered observation provides.

Declarations of Conflict of Interest The authors declared that they have no conflicts of interest in this work.

Data Availability The apparatus source code, all 408 recorded trajectories, observation lens outputs, analysis scripts, and evaluation prompt templates are publicly available as a replication package at https://doi.org/10.5281/zenodo.20337565. The package includes raw trajectory data, pre-computed projection results for all five lenses, and the scripts that produce every figure and table reported in this paper.

Usage of AI Tools AI-assisted tools (Claude - Anthropic, Grammarly, and Overleaf’s AI assistant features) were used during manuscript preparation for proofreading and readability improvements. Analysis scripts for trajectory statistics and visualization code for scientific figures were written with AI assistance under author-specified instructions and reviewed by the authors. The AI tools were not used to generate research ideas, design the study methodology, or produce images directly used in the paper. The authors take full responsibility for the content of this work. Editorial Policies for: Springer journals and proceedings: https://www.springer.com/gp/editorial-policies Nature Portfolio journals: https://www.nature.com/nature-research/editorial-policies Scientific Reports: https://www.nature.com/srep/journal-policies/editorial-policies BMC journals: https://www.biomedcentral.com/getpublished/editorial-policies

Springer Nature 2021 LATEX template 36

Projecting the Emerging Mindset of SWE Agent

ROLE FRAME

You are Ada, a meticulous and insightful code analyst renowned for your ability to delve into software intricacies; understand complex codebases, decipher algorithmic logic, and discern intent behind code.

identity analyst role and code-reading stance

TOOL USE

TOOLS

Tools are how you direct your attention, navigate the codebase, and articulate steps. You can use one tool per message. Use tools step-by-step, with each tool use informed by the result of the previous one. Tool use is formatted using XML-style tags:

turn grammar

<tool_name> <parameter>value</parameter> </tool_name>

dependence; XML parsing

specialized analytical tools to explore, understand, build a comprehensive model of the codebase: get_structural_overview create_memory

REPOSITORY

peek_code_region

explore_code

get_hint_on_snippet

stash_conclusion

Repository: flask Project Structure: flask/ README.md # 52 lines src/flask app.py # 1537 lines blueprints.py # 129 lines

OBJECTIVE

achieve a deep and thorough understanding of the codebase; break the analysis into manageable steps; employ one tool per message.

MEMORY: No memory has been created yet. BOOKMARK: No bookmarks have been created yet. CONCLUSION: No conclusion has been reached yet.

STATE CHANNELS

TERMINATION

bookmark

<stash_conclusion> formally record a significant understanding; indicate if analysis is complete or if you intend to continue. Set proceed_to_finalize=true when conclusions sufficiently address the task.

one action; previous-result

contact surface orientation, reading, traversal; memory, bookmark, hint, conclusion

world + task bounded repository world and code-understanding objective

working record memory, bookmark, conclusion sections

closure finalization is a tool-mediated decision

Fig. A1: Prompt scaffold of Ada as a repository-level code-understanding agent, organized and abstracted by functional components. The figure compresses the role frame, one-tool-per-turn protocol, tool interface, repository context, state channels, and explicit termination rule into a prompt-listing view.

Appendix A

Appendix: Experiment Pipeline

This appendix documents the experimental apparatus behind Ada and the CoachJudge condition. It is intended as replication-oriented scientific reporting: it describes what the apparatus asks the agent to do, what information the agent can observe, how the experimental conditions modify the agent’s operating stance, and how the coach-judge process enters a trajectory. It does not report behavioral findings.

A.1

Ada Apparatus

A.1.1

Prompt Scaffold

Ada is launched as a read-only code-understanding agent. The prompt scaffold gives it an identity, a constrained action protocol, a repository context, and a persistent state for accumulating understanding across turns. The scaffold is designed to make each trajectory observable: every turn contains stated reasoning, one selected action, and the environment’s response.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

37

Two exact prompt excerpts matter for interpreting the trajectory design. The first is the one-action-per-turn constraint. You must only use one tool per interaction, and your analysis must build up step-by-step from earlier results.

The second is the termination contract. A trajectory ends only when the agent uses the conclusion tool and explicitly marks the analysis as ready to finalize. - proceedtofinalize: (required) A boolean value (true or false). - Set to true if you believe this conclusion ... sufficiently addresses the code reading task, and you wish to finalize your analysis.

The prompt, therefore, turns code understanding into a bounded sequence of observable choices. The agent is not asked to answer immediately from prior knowledge. It is asked to reason, inspect the repository through the available tools, record intermediate understanding, and terminate only when it believes the task has been sufficiently addressed.

A.1.2

Task Setup Prompts

The system-side scaffold was paired with a short task block that specified what kind of repository understanding Ada should produce. These task prompts varied by Task Family, while the experimental conditions only changed Ada’s operating stance around the same underlying task. The prompts below reproduce the userfacing task setups in paper-facing terminology rather than internal run identifiers. Architecture Analysis. GLOBAL GOAL: Analyze the provided codebase to determine its overall architecture and architectural style. You should identify the major components, understand how they are organized, and describe the primary patterns of interaction between them. Your analysis should conclude with a summary of the architectural design. Constantly viewing the structure of code might be helpful. And creating memory on important logical and data flow will help you in concluding the architecture design. You can start now.

ASR Inference. GLOBAL GOAL: Analyze the provided codebase to identify its Architecturally Significant Requirements (ASRs). Investigate the code for evidence of design decisions that satisfy implicit requirements such as performance, security, scalability, or reliability. Your analysis should conclude with a list of inferred ASRs, each supported by specific code evidence. Constantly exploring the reference of code pieces might be helpful. Feel free to get a hint from the system. You can start now.

Springer Nature 2021 LATEX template 38

Projecting the Emerging Mindset of SWE Agent

Feature Implementation Tracing. GLOBAL GOAL: Your objective is to locate and describe the complete implementation of the target feature. Trace this feature from its most likely entry point through its core logic, data handling, and any interactions with other modules, concluding with a summary of the end-to-end implementation path. If you get stuck, try bookmarking or getting a hint from the system. Constantly getting structure overviews of files is recommended. Verify your memories and conclusions before stopping. You can start now.

For this Task Family, the repository-specific target feature was substituted into the prompt. The six targets were request routing and dispatching in Flask, session management and cookie persistence in Requests, the DecisionTreeClassifier implementation path in Scikit-learn, conditional edge traversal in Graphiti, the core prompt-optimization workflow in Prompt Optimizer, and the streaming inference pipeline in TensorZero. Risk and Code-Smell Review. GLOBAL GOAL: Act as a code reviewer and systematically analyze the provided codebase to identify potential risks, comprehension difficulties, or code smells. Your analysis should result in a small list of these issues, with each issue referencing the specific location in the code where it was found. Use bookmarks to flag code snippets that appear to be risky. You might want to scan the code files, so using peek region or exploring the code might be helpful. You can start now.

These task blocks were short by design. They specified the repositoryunderstanding objective and a small amount of task-local guidance, while the scaffold and condition overlays controlled the action protocol, memory surfaces, termination rule, and evidential stance.

A.1.3

Tool Surface

The seven tools define Ada’s contact surface with the repository and with its own accumulating state. Table A1 describes them by functional role rather than internal format. The navigation tools expose repository evidence; the cognitive tools externalize intermediate reasoning, unresolved questions, clarification requests, and final synthesis. Table A1: Ada tool signatures as exposed in the prompt schema. Tool

Group

peek code region Navigation

Parameters

Function in trajectory

file path: string; start line number: number; optional end line number: number.

Reads exact line ranges and records line-level file access.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

39

Tool

Group

Parameters

Function in trajectory

get structural overview

Navigation

file path: string; optional logical block name: string.

explore code

Navigation

action type: go to definition

Calls the structure summarizer for a file or named block and records structural access. Uses VS Code definition/reference providers or retrieves the next local block.

find all references

create memory

Cognitive

bookmark

Cognitive

get hint on snippet

Cognitive

stash conclusion Cognitive /

termination

go to next block; current file path: string; current line number: number; conditional symbol name: string. memory title: string; memory content: string; optional confidence level: high medium low. file path: string; line number: number; note: string.

file path: string; code range: string; description of confusion: string; optional what i tried or think: string. Optional title: string; content: string; scope description: string; proceed to finalize: boolean.

Stores intermediate hypotheses or synthesized understanding for later prompt reinjection. Marks a code location for later attention. The Cline-oriented prompt names the note field question or note; the Copilot schema names it note. Requests a bounded local hint for a confusing snippet. The hint generator is instructed not to answer the full task. Records a major or final conclusion. Finalization occurs when proceed to finalize is true, corresponding to u∗ in Equation 1.

This tool surface is intentionally narrower than a full developer environment. Ada can inspect, navigate, remember, bookmark, ask for local hints, and conclude. It cannot edit files or execute arbitrary exploratory routines. This keeps trajectories centered on repository understanding rather than repair or code generation.

A.1.4

Condition Variants

The four main experimental conditions preserve the same task set, repositories, tool surface, and read-only boundary. They differ only in the stance imposed on Ada while it conducts the trajectory. Table A2: Condition-level modifications to Ada’s operating stance. Condition

What changes

Rationale in the experiment

Natural Condition

Uses the base scaffold (role, tool definitions/examples, tool-use guidelines, repository context, accumulated memory/bookmarks/conclusions, objective, notes/history, and the current task). No additional pressure or evidential constraint is added. Adds an explicit exploration budget: each turn is framed as consuming budget, so Ada must decide what not to inspect. A budget-status block is shown before the objective (default target: 12 exploration turns, unless overridden). Budget reminders are injected as the remaining budget drops.

Establishes the unperturbed reference trajectory for the same agent, tasks, repositories, and tools.

Budget Pressure

Tests how the trajectory reorganizes when exploration space is scarce while the tool surface remains unchanged.

Springer Nature 2021 LATEX template 40

Projecting the Emerging Mindset of SWE Agent

Condition

What changes

Rationale in the experiment

Prior Suppression

Adds an evidence-first stance: prior/parametric familiarity is treated as weak intuition rather than evidence. When drawing conclusions, Ada must ground them in repository-specific observations (e.g., files, symbols, code paths, or tool-observed details) and clearly separate hypotheses from observed evidence. Applies both constraints: budget-aware exploration plus evidence-first grounding. Ada is asked to maximize task-relevant certainty under the remaining budget while avoiding unsupported prior-driven conclusions.

Tests whether the agent can separate likely prior familiarity from evidence gathered inside the current repository.

Combined Condition

Tests the joint constraint of scarce exploration and stricter evidential grounding.

Budget Pressure has both an initial prompt effect and a turn-time feedback effect. The initial prompt defines the budget as a real task constraint. During the trajectory, budget reminders are inserted at coarse milestones: around half the budget remaining, quarter budget remaining, three turns remaining, and the final turn. These reminders shift from narrowing the search space to consolidating and concluding from the strongest available evidence. Prior Suppression does not attempt to remove prior knowledge from the model. Instead, it changes the evidential rule of the task: prior familiarity may suggest hypotheses, but the reported conclusion should be justified by what Ada has observed in the repository during the trajectory.

A.1.5

Model Runtime and Context Handling

Several runtime settings matter for interpreting Ada’s trajectories. For the userconfigurable trajectory-generation models, Qwen3.5-397B-A17B and DeepSeekV3.2, the decoding temperature was fixed at 0.2. For the GitHub Copilot default models, GPT-4o and GPT-4.1, this parameter was not exposed by the runtime, so those trajectories inherit the provider’s default behavior. Model reasoning mode was turned off in the Ada apparatus because the ReAct-style agent design already includes a visible thinking segment as part of the trajectory itself; enabling an additional hidden reasoning mode would have mixed two different reasoning surfaces inside the same observation record. The context window was set to the largest capacity available for each model. When prompt assembly reaches that capacity, truncation is handled by the editor runtime’s internal priority logic rather than by a study-specific manual rule. The system prompts were given higher priority, while accumulated navigationturn messages were lower priority. As a result, when context overfill occurs, older or lower-priority trajectory material can be dropped, and some trajectories may therefore show goal drift that is partly induced by context saturation.

A.2

Coach-Judge Pipeline

A.2.1

Pipeline Architecture

The Coach-Judge condition adds a second reasoning process to the trajectory. Ada still performs one tool-mediated action per turn. After a turn is completed, the coach-judge process receives the completed turn and updates its trajectory

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

41

record. Specialized judges then evaluate the turn from different perspectives, and a coach decides whether the next observation should remain unchanged or include intervention. The functional flow is: Completed Turn → Trajectory Summary → Judge Panel → Aggregated Diagnostic Report → Coach Decision → Next Observation

This design corresponds to Equations 6–7: the judge panel observes the trajectory up to the current turn, the coach conditions on the panel’s diagnostic report and Ada’s current action, and the resulting decision changes the next observation only when intervention occurs. The runtime coach and judges use DeepSeek-V3 in this condition. This is separate from the post-hoc automatic adjudication lens discussed in Appendix C; that evaluation layer uses a separate GLM, MiniMax, and Kimi combination and is not the mechanism that intervenes during Ada’s trajectory. Across both the runtime Coach-Judge layer and the post-hoc adjudication layer, the judge models ran with model reasoning mode turned off, a temperature of 0.2, and an output limit of 2048 tokens. The reasoning mode was disabled because the judges were used as bounded assessment components under time and budget constraints rather than as open-ended deliberators.

A.2.2

Judge Panel Composition

The judge panel has six possible roles. Two roles are always active because they monitor the overall trajectory and the agent’s reasoning stance. The other roles are activated only when the current turn falls within their domain. This keeps each turn evaluated by broad trajectory monitors plus the relevant action-specific monitor. The upstream reason for this composition is the separation of concerns. Overall and bias judges watch the trajectory of health and reasoning stance across all turns. Navigation, cognitive, conclusion, and error judges each inspect the local type of work being attempted. The coach, therefore, receives a report that combines persistent process-level monitoring with action-specific diagnosis.

A.2.3

Judge Prompt Structure

The judge prompts share a common structure. A system-facing portion defines the judge’s role and rating dimensions. A task-facing portion injects the current goal, a compact trajectory history, the current thinking block, the current action, and any action-specific context. The judge returns a structured assessment with a short rationale and categorical ratings. The categories differ by judge’s role, as summarized in Figure A2. Before these judges run, each completed turn is summarized into a compact record of what Ada was trying to do, what action it took, and what information came back. The summaries form a rolling trajectory memory for the judge panel. This design avoids requiring every judge to reread the entire raw trajectory at every turn while still preserving recent context.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

42

shared prompt skeleton

role-specific deltas

common: goal + trace slice + current step

Prompt family Role: specialized judge over Ada’s trajectory Context: global goal / task recent trajectory summaries current thinking or action observation / error / submission Task: compare intent, evidence, result decide if this step helps the run Return JSON: { reasoning, role-specific fields }

always

Overall Judge

always

Bias Judge

Strategic Observer: monitor workflow. Context: last 7 steps; goal; thinking. Ask: progress / looping / assumptions. flow_state; laziness; suggestion

Metacognition Analyst: self-awareness. Context: goal; evidence seen; thinking. Ask: parametric / confirmation bias; confidence. confidence_level; detected_bias

repo action

state action

Navigation Judge

Cognitive Judge

Senior Code Reviewer: navigation logic. Context: goal; steps; intent; action; feedback. Ask: target reasonable? result useful? scent_rating; yield_rating

Epistemic Analyst: thinking quality. Context: goal; steps; observation; cognitive action. Ask: synthesis/parroting? risk valid? quality; risk_validity

on error

finalize

Conclusion Judge Rigorous Auditor: final conclusion. Context: goal; repo tree; visited paths; submission. Ask: enough evidence? unsupported claims? missed files? saturation; grounding; missed_files

Error Judge Resilience Evaluator: error recovery. Context: failed action; error; adjacent steps. Ask: understood? pivot/retry/loop? severity? diagnosis; recovery; severity

Fig. A2: Demonstration of the runtime judge prompts under the Coach-Judge condition. The prompt family shares a common skeleton over goal, trajectory slice, current step, and JSON reasoning output, while each judge specializes in the context slice and requested judgment for workflow progress, bias, navigation, cognitive state, conclusion, or error recovery.

A.2.4

Aggregation Logic

The judge assessments are not averaged into a numerical score. Each judge contributes a short rationale and categorical ratings. The aggregation step converts these role-specific assessments into a diagnostic report for the coach. The report preserves the identity of each judge’s role, its evaluation dimensions, and its reasoning. In effect, aggregation is a structured handoff from specialized observers to the coach, rather than a vote or scalar threshold. This matters experimentally because intervention is not triggered by a single universal metric. The coach receives a distribution of diagnostic signals. For example, a weak navigation signal can be read differently if the overall judge still sees progress, while a premature conclusion signal has a stronger implication for rejection. The aggregation format, therefore, supports the coach’s qualitative decision among PASS, HINT, and REJECT.

A.2.5

Coach Decision Policy

The coach is instructed to balance autonomy and rigor. Its default stance is nonintervention: if the judges report no serious problem, the trajectory should continue without alteration. The coach intervenes when multiple diagnostic signals indicate a harmful pattern, and it rejects conclusion attempts when the conclusion appears premature or unsupported.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

43

Table A3: Coach decision policy in the intervention condition. Decision

Experimental meaning

Effect on Ada’s next observation

PASS

The current turn does not warrant intervention. Ada’s autonomy is preserved. The trajectory shows a concerning but recoverable pattern, such as weak navigation plus stalled progress. Ada attempts to close with a conclusion judged premature or insufficiently grounded.

The next observation contains only the normal environment response. A concise coach message is appended as guidance. Ada may use or ignore it in the next turn.

HINT

REJECT

The coach message is returned as a rejection of the attempted closure, and the trajectory continues.

The coach message is visible to Ada only under HINT or REJECT. Under PASS, the Coach-Judge process observes the trajectory but does not alter it. The condition therefore changes the content of some observations, not the formal turn structure of the trajectory.

Appendix B

Appendix: Pilot Launches

Before scaling to the full experimental matrix, three pilot launches were conducted to choose the prompt framework and to check that Ada could produce finite, recordable code-understanding trajectories under the seven-tool interface described in Appendix A. The pilot launches used the same repository-level task and the same foundation model, GPT-4o, while varying only the surrounding prompt framework.

B.1

Prompt Framework Selection

The pilot compared three prompt-framework sources: a Copilot-style prompt from a public leaked-prompt collection, a Trae-style prompt from a public leaked-prompt collection, and a Cline-style prompt derived from an officially released open-source project. The pilot was not used to compare final answer quality. Its purpose was narrower: to determine whether each framework could sustain Ada’s tool-mediated trajectory, reach explicit termination, and produce a usable code-understanding account. All three preserved pilot trajectories reached an explicit conclusion action. The preserved records show different exploration volumes: Cline terminated after 28 recorded tool turns, Copilot after 33, and Trae after 58. The corresponding tool-call distributions are shown in Figure B3. The pilot therefore established finite, recordable trajectories, while the stricter 50-turn bound was adopted later for the main matrix. Cline was selected for the main experiment because it avoided reliance on leaked prompt material, produced the shortest preserved pilot while still retaining enough exploration detail, and used a broader action mix than Copilot: four plotted

Springer Nature 2021 LATEX template 44

Projecting the Emerging Mindset of SWE Agent

Fig. B3: Pilot tool-call distributions across the three prompt frameworks. All three launches used the same task, the same foundation model, and the same seven-tool Ada interface. The figure reports the number of tool calls by action type, showing that all frameworks produced finite trajectories while differing in exploration volume and cognitive-tool use.

tool types plus the terminal conclusion action, compared with Copilot’s two plotted tool types plus the terminal conclusion action. Table B4: Pilot framework comparison. Framework

Source status

Role in design decision

Copilot

Public leaked-prompt collection from late 2024. Public leaked-prompt collection from early 2025. Officially released open-source project.

Demonstrated that a short framework could sustain the tool protocol, but the preserved pilot used a narrower action mix, and the source was leaked material.

Trae

Cline

B.2

Demonstrated that the tool protocol could remain stable in a longer pilot, but was not selected because the source was leaked material and the trajectory was the longest of the three. Selected for the main experiment because it used an official source, had a detailed prompt structure, and produced the shortest preserved pilot while still using navigation, memory, and traversal actions before termination.

GLM Exclusion

The pilot phase also tested the GLM family as a possible trajectory-generation model. GLM was excluded from the main trajectory-generation matrix because its tool-call compliance was not reliable enough for recoverable trajectory recording. Observed failure modes included malformed tool calls, calls to tools outside Ada’s seven-tool interface, and loss of the required tool-call format within a trajectory. These failures made the affected pilot trajectories unsuitable as empirical objects for the observation lenses.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

45

The exclusion applies only to trajectory generation. GLM was retained in the post-hoc automatic adjudication layer, where it contributes as one of the judge models described in Appendix C. In that role, GLM does not need to sustain Ada’s multi-turn tool-use protocol; it evaluates completed trajectory materials under a separate adjudication design.

Appendix C

LLM–Human Alignment and Adjudication Calibration

This appendix documents the post-hoc adjudication lens used in Section 3.4. The purpose is not to treat the automatic judges as a human substitute. The purpose is to make their protocol, internal agreement, human alignment, and length robustness visible enough that the Lens 3 results can be read with the right calibration.

C.1

Evaluation Protocol and Judge Prompts

The adjudication layer evaluates completed Ada trajectories through five evaluation families. Task Relevance, Trace Grounding, and Termination Calibration operate at the conclusion level: they ask whether the final conclusion addresses the task, whether it is grounded in the observed trajectory, and whether the stopping point was well calibrated. Navigation Prior Usage and Reasoning Prior Usage operate at the Turn or artifact level: Navigation Prior Usage evaluates whether navigation moves are led by evidence already visible in the trajectory, while Reasoning Prior Usage evaluates whether cognitive artifacts are organized by trajectory evidence or by prior-shaped framing. The three conclusion-level evaluation families use a three-model vote over GLM-5, Kimi-K2.5, and MiniMax-M2.7. The two Prior Usage families use a single judge model. Evaluation items are anonymized with respect to the trajectorygenerating model and experimental condition, and judge models are selected to avoid direct overlap with trajectory-generation models where possible. Each conclusion-level family contains 407 usable judged items, composed of 383 maingrid items and 24 Coach-Judge items; one additional candidate item was excluded after repeated server-side rejection. The Prior Usage evaluations contain 7,193 Navigation Prior Usage judgments and 929 Reasoning Prior Usage judgments. These judge runs used the same runtime settings as the Coach-Judge layer: model reasoning mode was turned off, temperature was fixed at 0.2, and outputs were capped at 2048 tokens. The three conclusion-level evaluations use four ordered labels, with occasional unclear or review flags handled outside the ordinal scale. The Prior Usage evaluations use three ordered evidence-orientation labels plus unclear: evidence-led or grounded maps to 0.0, mixed maps to 0.5, and prior-led or prior-shaped maps to 1.0 in the prior-leaning index. Figure C4 summarizes the prompt structure without reproducing full prompt text.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

46

Conclusion-level judges

Shared structure System role: single evaluation duty no outside repository knowledge structured response only Common inputs: Task Family task prompt conclusion title/content optional scope note Output pattern: rationale one ordinal label short evidence lists

Termination calibration

Conclusion grounding

Task relevance Question: Does the conclusion address the requested task?

Question: Is the conclusion supported by the observed trajectory?

Extra inputs: required concerns optional concerns off-task patterns

Extra inputs: prefix summary trajectory excerpt viewed-snippet summary

Requested judgment: task-addressing label covered concerns missed concerns

Requested judgment: support label supported claims under-supported claims

Question: Was this a defensible moment to stop? Extra inputs: global trajectory summary coverage summary repository-structure summary Requested judgment: stop-calibration label scope signals possible missing regions

Turn-level judges Shared structure

Navigational orientation

Synthesis orientation

System role: judge source of orientation or organization plausible is not automatically evidence-led no outside repository knowledge

Evaluation target: one repository-navigation action

Evaluation target: one cognitive artifact

Extra inputs: turn index tool name and summary action text

Extra inputs: artifact type, title, content scope description artifact action

Common inputs: Task Family and task repository name prefix summary local interaction window

Requested judgment: evidence-led / mixed / prior-led / unclear evidence and prior-use indicators

Requested judgment: trajectory-grounded / mixed / priorshaped / unclear trajectory and prior-shape indicators

Fig. C4: Structural templates for the post-hoc judge prompts. The diagrams show the shared input scaffold, the per-family evaluation target, and the requested output shape. They omit full prompt wording; the released prompt templates provide the exact wording for replication.

Table C5: Three-judge agreement for conclusion-level adjudication. The highlighted row marks the weakest agreement surface. Evaluation family Task Relevance Trace Grounding Termination Calibration

C.2

n

Pair exact

Mean step diff.

All-same rate

407 407 407

0.607 0.622 0.440

0.483 0.387 0.645

0.452 0.435 0.204

Inter-Judge Agreement

Table C5 reports agreement among the current three conclusion-level judges. Task Relevance and Trace Grounding have moderate internal agreement. Termination Calibration is the reliability bottleneck: pairwise exact agreement is lower, ordinal disagreement is larger, and all three judges select the same label on only about one fifth of items.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

47

Table C6: Judge severity audit for Termination Calibration. The highlighted row marks the severity outlier. Judge

Mean Termination Calibration score

DeepSeek-V3.2 Kimi-K2.5 Qwen3.5-397B-A17B GLM-5 MiniMax-M2.7

3.646 3.292 3.209 3.172 2.631

Table C7: Human–LLM alignment by evaluation family. Task Relevance is highlighted as the strongest alignment; Trace Grounding and Reasoning Prior Usage are italicized as the weakest human-mirroring surfaces. Eval

n Pearson r Spearman ρ Weighted κ Exact Within 1 Mean diff.

Task Relevance Trace Grounding Termination Calibration Navigation Prior Usage Reasoning Prior Usage

50 50 50 50 50

0.608 -0.135 0.318 0.282 0.077

0.616 -0.089 0.297 0.312 0.150

0.547 0.116 0.237 0.305 0.106

44% 54% 40% 48% 42%

84% 98% 88% 80% 80%

+0.252 +0.264 -0.068 -0.240 +0.160

The lower reliability of Termination Calibration is partly explained by severity differences. Table C6 reports a broader judge-severity audit for Termination Calibration. MiniMax-M2.7 is much more conservative than the other judges, with a mean Termination Calibration score of 2.631 on the 1–4 scale. Termination Calibration should therefore be read as a projected stopping-maturity signal, not as a mechanically obvious endpoint metric.

C.3

Human Alignment Check

A separate human-alignment check sampled 250 items, with 50 items per evaluation family. One annotator supplied labels under the same information regime used for the automatic judges. Annotation notes report approximately 67 minutes of labeling decisions for about 150 items, which is itself informative: some dimensions, especially Trace Grounding, require cross-checking the conclusion against a full trajectory and are difficult under bounded human annotation time. Table C7 compares the sampled human labels against the LLM aggregate labels or scores. Task Relevance is the strongest human-aligned surface. Termination Calibration and Navigation Prior Usage are directionally useful but noisy. Trace Grounding and Reasoning Prior Usage are the weakest human-mirroring surfaces in this sample. Table C8 compares model–model alignment with human–model alignment on the three conclusion-level families. The Trace Grounding gap is the clearest warning sign: judge models agree with one another more than they agree with the human annotator.

Springer Nature 2021 LATEX template 48

Projecting the Emerging Mindset of SWE Agent

Table C8: Model–model alignment versus human–model alignment for the three conclusion-level families. The bold Trace Grounding human-model values mark the critical alignment gap. Model–model r Human–model r Model–model κ Human–model κ

Eval Task Relevance Trace Grounding Termination Calibration

C.4

0.854 0.318 0.458

0.571 -0.089 0.243

0.839 0.269 0.396

0.520 -0.077 0.224

Interpretation

Task Relevance is the most trustworthy automatic adjudication surface in the current validation. Its Pearson correlation with the human labels is 0.608, and its weighted κ is 0.547. Main-text claims that rely on whether a conclusion addressed the task therefore rest on the firmest Lens 3 ground. Trace Grounding and Reasoning Prior Usage should be read differently. They are machine-consensus projections, not human proxies. The judges can process the whole trajectory without fatigue, while a bounded human annotation session must cross-reference conclusions, intermediate observations, and prior-shaped reasoning under time pressure. The present data cannot distinguish whether model agreement reflects shared model priors or better large-context cross-referencing. What it establishes is more limited and more useful: Trace Grounding and Reasoning Prior Usage gain credibility when they converge with other observation lenses, not when treated as standalone human-aligned measures. Termination Calibration and Navigation Prior Usage occupy a middle ground. Their correlations with human labels are positive, but weak enough that the paper should lean on label-composition patterns rather than small mean-score differences. For Termination Calibration, this means reading shifts such as ready-to-finalize versus borderline as qualitative stopping profiles. For Navigation Prior Usage, it means using the prior-leaning index as a directional signal about navigation orientation, not as a precise measure of hidden intent. The alignment check also reveals an asymmetry in evaluation difficulty. Dimensions that compare a conclusion against the task are more human-accessible than dimensions that require checking a conclusion against the full trajectory or inferring prior familiarity from behavior. This is not only a limitation of the validation exercise; it is also a result about which evaluation questions are tractable for human annotation at scale.

C.5

Length Robustness Checks

Table C9 reports the conclusion-length robustness check for the main non-coach grid. Raw correlations are non-trivial, especially for Task Relevance and Trace Grounding, but the added explanatory power after controlling for model, condition, Task Family, and Repository is small. The corresponding visual diagnostic for conclusion length and Task Relevance appears in Figure D9. Table C10 reports the analogous trajectory-length check for Termination Calibration.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

49

Table C9: Conclusion length versus conclusion-level adjudication scores, main noncoach grid. Evaluation family Task Relevance Trace Grounding Termination Calibration

n

Raw Pearson r

Added ∆R2

383 383 383

0.533 -0.316 0.300

0.016 0.004 0.001

Table C10: Trajectory length versus Termination Calibration, main non-coach grid. Metric Valid tool turns

n

Raw Pearson r

Raw R2

Added ∆R2

383

0.258

0.067

0.002

Table C11: Conclusion length by trajectory-generating model, main non-coach grid. Model

n

Mean words

Median words

Qwen3.5-397B DeepSeek-V3.2 GPT-4o GPT-4.1

95 96 96 96

323.1 254.7 125.0 97.0

311.0 246.5 124.0 89.5

The model-level length table explains why raw correlations appear in the first place. Conclusion length is strongly entangled with the trajectory-generating model. Qwen3.5-397B and DeepSeek-V3.2 produce much longer conclusions than GPT-4.1 and GPT-4o, so a raw length association can partly reflect model identity rather than an independent length effect. Length is therefore a visible proxy for answer development, but not an independent driver of judge scores in the controlled checks. The automatic adjudication lens should be read as a set of semantic projections conditioned by model, task, condition, and repository, rather than as a disguised length measure.

Appendix D

Supplementary Results and Unreported Details

This appendix reports selected supplementary materials that support claims compressed in Section 4. The materials are organized by observation lens and are included only where they clarify a specific claim, caveat, or extension.

D.1

Tool-Use Concentration and Task Variation

The surface activity view shows that Ada’s visible tool use is highly concentrated. Figure D5 reports the overall tool-call distribution and its variation by Task Family.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

50

PKR + OVW = 81.7%

80

3000

Tool calls

100

60 2000 40 1000

20

0

Cumulative share (%)

4000

Avg calls / trajectory

12 10 8 6 4 2 0 Architecture

0 PKR

OVW

EXP

MEM

CON

BKM

PKR

HNT

(a) Overall concentration.

ASR inference Feature tracing OVW

EXP

MEM

CON

Risk review BKM

HNT

(b) Variation by Task Family.

Fig. D5: Supplementary surface view of Ada’s tool use. Panel (a) shows tool-call concentration across the full main matrix. Panel (b) shows average tool-call composition by Task Family, indicating that task openings modulate the balance of structural orientation, close reading, traversal, and cognitive actions.

Architecture

14

21

19

17

ASR inference

13

21

20

17

Feature tracing

19

Risk review

17

0

19 14

25

20 21

14

13

18

50

75

100

Share of action-policy tags (%) close read traverse trace

zoom in skim zoom out

dump resume cognitive streak

interim synth close out

Fig. D6: Task Family changes the transition mix even under the same apparatus. Stacked bars report the share of adjacent-action transition tags, including close read, traverse, trace, zoom in, skim, dump, cognitive streak, interim synthesis, and close out.

Across the main matrix, close code reading and structural overview account for most actions. Task Family changes the balance of those actions: feature-implementation tasks increase close reading, while broader repository-understanding openings retain more structural orientation.

D.2

Extended Action-Policy Projections

Figure D6 extends the action-policy lens used in the main text. The Combined Condition is included here because the main Results figure omits it for compactness. Across models, Combined Condition profiles closely follow the pressure-family pattern rather than the Natural or Prior Suppression pattern. Task Family also shifts action policy: Feature Implementation Tracing increases close read and traverse

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent -1.4

-5.4

+0.3

+2.0

+0.8

+2.0

+0.2

+1.1

+0.0

+0.0

GPT-4o +8.7 +0.9 -3.1

-5.1

-8.0

-0.4

+3.7

+2.8

-0.2

+0.9

-0.2

+0.0

+0.0

DeepSeek-V3.2 -8.7 +0.0 +4.5 +2.9 +4.2 +0.6 -0.9

-1.4

+0.1

-0.3

-1.0

+0.0

+0.0

-2.2

-1.9

-0.8

+0.1

+0.0

+0.0

5

0

−5

Qwen3.5-397B

-2.0

s clo

-1.4

+0.6

+3.6

+9.2

-0.5

-4.7

Deviation from average (pp)

GPT-4.1 +2.0 +0.4 -2.0

51

t t r k e e d in kim ou ump ume trea nth e ou read the ea vers trac om s o sy s d r om re ve s rim zo los the tra c zo i o e it int gn co

er

Architecture -1.6

-4.0

+0.5

+1.5

+3.0

+0.4

+1.1

-0.4

-0.6

-0.2

+0.4

+0.0

+0.0

4

ASR inference -3.0

-2.1

+0.1

+2.1

+2.8

-0.4

+0.5

-0.5

+0.6

+0.3

-0.4

+0.0

+0.0

Feature tracing +3.4 +5.5 -0.5

-3.8

-0.8

+0.2

-3.0

-0.6

-0.5

-0.4

+0.5

+0.0

+0.0

Risk review +1.2 +0.6 -0.1 +0.3 -5.0

-0.2

+1.5

+1.4

+0.5

+0.3

-0.4

+0.0

+0.0

2 0 −2 −4

Deviation from average (pp)

Fig. D7: Model-level deviations from the average action-policy profile. Positive values indicate that a model uses a transition tag more than the cross-model average, and negative values indicate lower-than-average use.

t t r k e e d in kim ou ump ume trea nth e ou read the ea vers trac om s o sy s d om os ther re ve s rim zo l tra c zo o e iti int gn co

er

s clo

Fig. D8: Task-Family deviations from the average action-policy profile. Positive values indicate that a Task Family uses a transition tag more than the cross-task average, and negative values indicate lower-than-average use. This figure supplies the relative-to-average task view omitted from the main text.

shares, while Architecture Analysis and ASR Inference retain more trace and skim behavior. The model-deviation heatmap in Figure D7 gives a relative view of the same action-policy lens. It centers each transition tag against the cross-model average, making model-characteristic signatures visible: GPT-4o is close-read heavy, DeepSeek-V3.2 is trace-heavy, and Qwen3.5-397B is skim-heavy. This view supplements the stacked bars by showing deviation from the common baseline rather than absolute composition. Figure D8 adds the matching relative-to-average view for Task Family. Feature Implementation Tracing over-indexes on close read and traverse while underindexing on skim, making its implementation-following stance visible in a single panel. Architecture Analysis and ASR Inference tilt more toward skim and zoom-in

Springer Nature 2021 LATEX template 52

Projecting the Emerging Mindset of SWE Agent

T1 score (normalised)

4.0 3.5 3.0 2.5 2.0

Pearson r = +0.52 Added R² (controls) = +0.012

1.5

GPT-4.1 GPT-4o

1.0 200

400

600

DeepSeek-V3.2 Qwen3.5-397B

800

Conclusion length (words)

Fig. D9: Conclusion length versus Task Relevance. The pooled association is positive, but the diagnostic should be read cautiously: length may correlate with fuller task coverage without guaranteeing stronger grounding.

behavior, while Risk and Code-Smell Review carries the clearest interim-synthesis and dump deviations. This heatmap supports the same claim as Figure D6, but in deviation form rather than composition form.

D.3

Supplementary Adjudication Diagnostic

Table D12: Conclusion-length diagnostic by model. Task Relevance is reported on the original 1–4 adjudication scale. Model

DeepSeek-V3.2 GPT-4.1 GPT-4o Qwen3.5-397B

Traj.

Median words

Mean Task Relevance

Length–Task Relevance r

96 96 96 96

262.0 102.5 134.0 341.0

3.719 3.035 1.597 3.719

0.115 0.305 0.230 0.372

Conclusion length is reported here as a diagnostic, not as a quality measure. Longer conclusions can appear more complete, but length does not by itself establish repository grounding. Figure D9 shows a moderate positive association between conclusion length and Task Relevance in the pooled data, while the added explanatory value after controls is small. Table D12 reports the compact model-level summary. Overall, the pooled length–Task Relevance correlation is r = 0.521, while the controlled added R2 reported in the diagnostic figure is 0.012. The appendix therefore treats length as a cautionary surface diagnostic rather than as a substitute for the adjudication lens.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

D.4

53

Repository-Footprint Anchors and Transition Corridors

Table D13: Recurring anchor files by Repository, ranked by average position across conditions. Repository

Top recurring anchors

Flask

src/flask/app.py; src/flask/sansio/app.py; src/flask/sansio/scaffold.py graphiti core/graphiti.py; graphiti core/search/search.py; graphiti core/edges.py middleware.js; packages/core/src/services/llm/service.ts; packages/core/src/services/prompt/service.ts src/requests/sessions.py; src/requests/ init .py; src/requests/adapters.py sklearn/tree/ classes.py; sklearn/base.py; sklearn/tree/ tree.pyx tensorzero-core/src/endpoints/inference.rs; gateway/src/main.rs; tensorzero-core/src/config parser.rs

Graphiti Prompt Optimizer Requests Scikit-learn TensorZero

Table D14: Top file-to-file transition corridors by Repository. Repository

Corridor

Flask

src/flask/app.py → src/flask/sansio/app.py graphiti core/graphiti.py → graphiti core/search/search.py packages/core/src/services/prompt/service.ts

Graphiti Prompt Optimizer Requests Scikit-learn TensorZero

Moves

Traj.

20

17

12

11

9

9

→

packages/core/src/services/template/processor.ts src/requests/sessions.py → 16 src/requests/adapters.py sklearn/ init .py → sklearn/base.py 8 tensorzero-core/src/lib.rs → 10 tensorzero-core/src/endpoints/mod.rs

14 8 10

The footprint lens can be summarized without showing another full heatmap. Table D13 reports recurring anchor files: files that repeatedly appear among the most visited files for a repository across conditions. Table D14 reports the most frequent file-to-file movement corridors. These tables expose recurring spatial attention in the observed trajectories; they do not claim that the files are objectively the most important files in the repository.

D.5

Coach-Judge Condition Details

Springer Nature 2021 LATEX template 54

Projecting the Emerging Mindset of SWE Agent

Table D16: Per-tool Turn shifts under Natural and Coach-Judge conditions for GPT-4.1. PKR = peek code region; OVW = get structural overview; EXP = explore code. Tool

Natural mean

Coach mean

Mean difference

Sign-test p

PKR OVW EXP

10.58 3.00 2.79

19.08 13.71 3.29

+8.50 +10.71 +0.50

0.0227 < .001 1.0000

Table D17: Trace Grounding label distribution under Natural and Coach-Judge conditions. Trace Grounding label

Natural

Coach-Judge

Well supported Partially supported Thinly supported

10 (41.7%) 14 (58.3%) 0 (0.0%)

3 (12.5%) 19 (79.2%) 2 (8.3%)

Table D15: Paired Natural versus Coach-Judge surface/action statistics for GPT-4.1. Metric Turns Elapsed time (s) Conclusion attempts Error turns Coach-message turns

Natural mean

Coach mean

Mean difference

Sign-test p

18.8 94.4 0.5

48.1 286.2 8.3

+29.3 +191.8 +7.8

< .001 .0015 < .001

0.3 0.0

4.6 11.2

+4.3 +11.2

< .001 < .001

Table D15 reports paired supplementary numbers for GPT-4.1 under Natural and Coach-Judge conditions. The table documents the scale of added interaction without repeating the full RQ3 interpretation. Coach-Judge trajectories contain more turns, more conclusion attempts, more error turns, and direct coach messages in a substantial share of turns. Table D16 adds the missing per-tool breakdown for the same paired comparison. The largest absolute shift is in structural overview activity, while explore-code usage is nearly flat. The added interaction therefore looks less like a new traversal strategy and more like repeated re-orientation plus additional close reading. The direct coach messages are split into 179 hint turns and 39 rejection turns among 269 coach-message turns. This distribution supports the main-text reading that the intervention condition often extended trajectories through guidance and conclusion rejection rather than through a change to the formal turn structure. At the tool level, the same pattern appears numerically: PKR rises from 10.58 to 19.08 mean turns, OVW rises from 3.00 to 13.71, and EXP changes only from 2.79 to 3.29.

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

55

Table D17 reports the Trace Grounding label distribution for the same paired comparison. The Coach-Judge condition reduces the share of well-supported conclusions and introduces thinly supported cases. This table gives the categorical detail behind the main RQ3 adjudication panel.

References [1] Liu, S., Chen, Y., Krishna, R., Sinha, S., Ganhotra, J., Jabbarvand, R.: Processcentric analysis of agentic software systems. Proceedings of the ACM on Programming Languages 10(OOPSLA1), 1961–1988 (2026). https://doi.org/10. 1145/3798271 [2] Desmond, M., Lee, J.Y., Ibrahim, I., Johnson, J.M., Sil, A., MacNair, J., Puri, R.: Agent trajectory explorer: Visualizing and providing feedback on agent trajectories. Proceedings of the AAAI Conference on Artificial Intelligence 39(28), 29634–29636 (2025). https://doi.org/10.1609/aaai.v39i28.35350 [3] Ou, T., Guo, W., Gandhi, A., Neubig, G., Yue, X.: AgentDiagnose: An open toolkit for diagnosing LLM agent trajectories. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 207–215. Association for Computational Linguistics, Suzhou, China (2025). https://doi.org/10.18653/v1/2025.emnlp-demos.15 [4] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y.: ReAct: Synergizing reasoning and acting in language models. arXiv (2023). https:// doi.org/10.48550/arXiv.2210.03629 [5] Contributors, C.: Cline: Autonomous coding agent. https://github.com/cline/ cline. Open-source project, Apache 2.0 License (2024) [6] Bourque, P., Fairley, R.E. (eds.): SWEBOK: Guide to the Software Engineering Body of Knowledge, Version 3.0 edn. IEEE Computer Society, Los Alamitos, CA (2014) [7] Simon, H.A.: A behavioral model of rational choice. The Quarterly Journal of Economics 69(1), 99–118 (1955) https://arxiv.org/abs/1884852. https://doi.org/ 10.2307/1884852 [8] Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K.R., Yao, S.: Reflexion: Language agents with verbal reinforcement learning. In: Thirty-Seventh Conference on Neural Information Processing Systems (2023) [9] Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C.A., Jia, H., Travers, A., Zhang, B., Lie, D., Papernot, N.: Machine unlearning. In: 2021 IEEE Symposium on Security and Privacy (SP), pp. 141–159. IEEE, San Francisco, CA, USA (2021). https://doi.org/10.1109/SP40001.2021.00019

Springer Nature 2021 LATEX template 56

Projecting the Emerging Mindset of SWE Agent

[10] Bouzenia, I., Pradel, M.: Understanding software engineering agents: A study of thought-action-result trajectories. In: Proceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering (ASE) (2025). arXiv:2506.18824 [11] Wang, X., Wei, J., Schuurmans, D., Le, Q.V., Chi, E.H., Narang, S., Chowdhery, A., Zhou, D.: Self-consistency improves chain-of-thought reasoning in language models. In: The Eleventh International Conference on Learning Representations (2022) [12] Zhang, K., Yao, W., Liu, Z., Feng, Y., Liu, Z., N, R.R., Lan, T., Li, L., Lou, R., Xu, J., Pang, B., Zhou, Y., Heinecke, S., Savarese, S., Wang, H., Xiong, C.: Diversity empowers intelligence: Integrating expertise of software engineering agents. In: The Thirteenth International Conference on Learning Representations (2024) [13] Arcuschin, I., Janiak, J., Krzyzanowski, R., Rajamanoharan, S., Nanda, N., Conmy, A.: Chain-of-thought reasoning in the wild is not always faithful. In: Workshop on Reasoning and Planning for Large Language Models (2025) [14] Zhou, S., Ling, R., Chen, J., Wang, X., Fan, T., Wang, H.: When more thinking hurts: Overthinking in LLM test-time compute scaling. arXiv (2026). https:// doi.org/10.48550/arXiv.2604.10739 [15] Jimenez, C., Lieret, K., Narasimhan, K., Press, O., Wettig, A., Yang, J., Yao, S.: SWE-agent: Agent-computer interfaces enable automated software engineering. In: Advances in Neural Information Processing Systems 37, pp. 50528– 50652. Neural Information Processing Systems Foundation, Inc. (NeurIPS), Vancouver, BC, Canada (2024). https://doi.org/10.52202/079017-1601 [16] Gandhi, S., Tsay, J., Ganhotra, J., Kate, K., Rizk, Y.: When agents go astray: Course-correcting SWE agents with PRMs. In: Workshop on Scaling Environments for Agents (2025) [17] Fan, Z., Vasilevski, K., Lin, D., Chen, B., Chen, Y., Zhong, Z., Zhang, J.M., He, P., Hassan, A.E.: SWE-Effi: Re-evaluating software AI agent system effectiveness under resource constraints. arXiv:2509.09853 (2025) [18] Li, H., Mang, Q., He, R., Zhang, Q., Mao, H., Chen, X., Zhou, H., Cheung, A., Gonzalez, J.E., Stoica, I.: Continuum: Efficient and robust multi-turn LLM agent scheduling with KV cache time-to-live. In: ICLR 2026 Workshop on Lifelong Agents: Learning, Aligning, Evolving (2026) [19] Huang, P., Guo, C., Zhou, L., Lorch, J.R., Dang, Y., Chintalapati, M., Yao, R.: Gray failure: The Achilles’ heel of cloud-scale systems. In: Proceedings of the 16th Workshop on Hot Topics in Operating Systems, pp. 150–155. ACM, Whistler BC Canada (2017). https://doi.org/10.1145/3102980.3103005

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

57

[20] Denison, C., MacDiarmid, M., Barez, F., Duvenaud, D., Kravec, S., Marks, S., Schiefer, N., Soklaski, R., Tamkin, A., Kaplan, J., Shlegeris, B., Bowman, S.R., Perez, E., Hubinger, E.: Sycophancy to subterfuge: Investigating rewardtampering in large language models (2024). https://arxiv.org/abs/2406.10162 [21] Augenstein, I.: Understanding the interplay between LLMs’ utilisation of parametric and contextual knowledge: A keynote at ECIR 2025. arXiv:2603.09654 (2026) [22] Pappu, A., El, B., Cao, H., di Nolfo, C., Sun, Y., Cao, M., Zou, J.: Multi-agent teams hold experts back (2026). https://arxiv.org/abs/2602.01011 [23] Sartori, C.C.: The specification gap: Coordination failure under partial knowledge in code agents (2026). https://doi.org/10.13140/RG.2.2.14475.96802 [24] Maryanskyy, A., Budnikov, D., Kaliyev, A.T.: When agents disagree: The selection bottleneck in multi-agent LLM pipelines. Applied Sciences 16(10), 4914 (2026). https://doi.org/10.3390/app16104914 [25] Barrak, A.: Traceability and accountability in role-specialized multi-agent LLM pipelines. 2025 40th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW), 315–322 (2025). https://doi.org/10. 1109/ASEW67777.2025.00064 [26] Kim, W., Park, S., In, Y., Kim, S., Lee, D., Park, C.: Beyond the final answer: Evaluating the reasoning trajectories of tool-augmented agents. arXiv:2510.02837 (2025) [27] Gautam, D., Garg, S., Jang, J., Sundaresan, N., Moghaddam, R.Z.: RefactorBench: Evaluating stateful reasoning in language agents through code. In: The Thirteenth International Conference on Learning Representations (2024) [28] Yang, J., Lieret, K., Jimenez, C.E., Wettig, A., Khandpur, K., Zhang, Y., Hui, B., Press, O., Schmidt, L., Yang, D.: SWE-smith: Scaling data for software engineering agents. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2025) [29] Ni, Z., Wang, H., Zhang, S., Lu, S., He, Z., You, W., Tang, Z., Du, Y., Sun, B., Liu, H., Hu, S., Chen, R., Li, B., Li, X., Hu, C., Jiao, B., Jiang, D., Lyu, P.: GitTaskBench: A benchmark for code agents solving real-world tasks through code repository leveraging (2025). https://arxiv.org/abs/2508.18993 [30] Rashid, M.S., Bock, C., Zhuang, Y., Buchholz, A., Esler, T., Valentin, S., Franceschi, L., Wistuba, M., Sivaprasad, P.T., Kim, W.J., Deoras, A., Zappella, G., Callot, L.: SWE-PolyBench: A multi-language benchmark for repository-level evaluation of coding agents. arXiv:2504.08703 (2025)

Springer Nature 2021 LATEX template 58

Projecting the Emerging Mindset of SWE Agent

[31] Lu, J., Holleis, T., Zhang, Y., Aumayer, B., Nan, F., Bai, H., Ma, S., Ma, S., Li, M., Yin, G., Wang, Z., Pang, R.: ToolSandbox: A stateful, conversational, interactive evaluation benchmark for LLM tool use capabilities. In: Chiruzzo, L., Ritter, A., Wang, L. (eds.) Findings of the Association for Computational Linguistics: NAACL 2025, pp. 1160–1183. Association for Computational Linguistics, Albuquerque, New Mexico (2025). https://doi.org/10.18653/v1/2025.findings-naacl. 65 [32] Yao, S., Shinn, N., Razavi, P., Narasimhan, K.R.: τ -bench: A benchmark for Tool-Agent-User interaction in real-world domains. In: The Thirteenth International Conference on Learning Representations (2024) [33] Wang, H., Huang, W., Wang, Y., Xi, Y., Lu, J., Zhang, H., Hu, N., Liu, Z., Pan, J.Z., Wong, K.-F.: Rethinking stateful tool use in multi-turn dialogues: Benchmarks and challenges. In: Che, W., Nabende, J., Shutova, E., Pilehvar, M.T. (eds.) Findings of the Association for Computational Linguistics: ACL 2025, pp. 5433–5453. Association for Computational Linguistics, Vienna, Austria (2025). https://doi.org/10.18653/v1/2025.findings-acl.284 [34] Liu, T., Xu, C., McAuley, J.: RepoBench: Benchmarking repository-level code auto-completion systems. In: The Twelfth International Conference on Learning Representations (2023) [35] Ding, Y., Wang, Z., Ahmad, W., Ding, H., Tan, M., Jain, N., Ramanathan, M.K., Nallapati, R., Bhatia, P., Roth, D., Xiang, B.: CrossCodeEval: A diverse and multilingual benchmark for cross-file code completion. In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. (eds.) Advances in Neural Information Processing Systems, vol. 36, pp. 46701–46723. Curran Associates, Inc., Red Hook, NY (2023) [36] Bairi, R., Sonwane, A., Kanade, A., C., V.D., Iyer, A., Parthasarathy, S., Rajamani, S., Ashok, B., Shet, S.: CodePlan: Repository-level coding using LLMs and planning. Proceedings of the ACM on Software Engineering 1(FSE), 675–698 (2024). https://doi.org/10.1145/3643757 [37] Du, J., Liu, Y., Guo, H., Wang, J., Huang, H., Ni, Y., Li, Z.: DependEval: Benchmarking LLMs for repository dependency understanding. In: Che, W., Nabende, J., Shutova, E., Pilehvar, M.T. (eds.) Findings of the Association for Computational Linguistics: ACL 2025, pp. 7150–7179. Association for Computational Linguistics, Vienna, Austria (2025). https://doi.org/10.18653/v1/2025. findings-acl.373 [38] Peng, W., Shi, Y., Wang, Y., Zhang, X., Shen, B., Gu, X.: SWE-QA: Can language models answer repository-level code questions? (2026). https://arxiv.org/abs/ 2509.14635 [39] Murphy, G.C., Notkin, D., Sullivan, K.: Software reflexion models: Bridging the

Springer Nature 2021 LATEX template Projecting the Emerging Mindset of SWE Agent

59

gap between source and high-level models. In: Proceedings of the 3rd ACM SIGSOFT Symposium on Foundations of Software Engineering, pp. 18–28. ACM, Washington D.C. USA (1995). https://doi.org/10.1145/222124.222136 [40] Ducasse, S., Pollet, D.: Software architecture reconstruction: A processoriented taxonomy. IEEE Transactions on Software Engineering 35(4), 573–591 (2009). https://doi.org/10.1109/TSE.2009.19 [41] Kazman, R., Klein, M., Clements, P.: ATAM: Method for architecture evaluation. Technical report, Defense Technical Information Center, Fort Belvoir, VA (August 2000). https://doi.org/10.21236/ADA382629 [42] Chen, L., Ali Babar, M., Nuseibeh, B.: Characterizing architecturally significant requirements. IEEE Software 30(2), 38–45 (2013). https://doi.org/10.1109/MS. 2012.174 [43] Eisenbarth, T., Koschke, R., Simon, D.: Locating features in source code. IEEE Transactions on Software Engineering 29(3), 210–224 (2003). https://doi.org/ 10.1109/TSE.2003.1183929 [44] Fagan, M.E.: Design and code inspections to reduce errors in program development. IBM Systems Journal 15(3), 182–211 (1976). https://doi.org/10.1147/sj. 153.0182 [45] Bacchelli, A., Bird, C.: Expectations, outcomes, and challenges of modern code review. In: 2013 35th International Conference on Software Engineering (ICSE), pp. 712–721. IEEE, San Francisco, CA, USA (2013). https://doi.org/10.1109/ ICSE.2013.6606617

Related documents

• The Complexity Kink: A Prompt-Side Structural Complexity Index for Code-Generation Reliability#978496
• Semantic Layer Induction from Raw Telemetry via Hierarchical LLM and RAG Abstraction#978497
• The Output-Space Hypothesis: Enumerative Equivalence Checking for Tensor Programs#978498
• DeltaSelect: Affordable A/B Testing for Coding Agents#978499
• Code-as-Auditor: Executable Compliance Reasoning via Regulation-to-Code#978500