GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices Yifang Tian

Yaming Liu

Zichun Chong

University of Toronto Toronto, Canada yifang.tian@mail.utoronto.ca

University of Toronto Toronto, Canada yaming.liu@mail.utoronto.ca

University of Toronto Toronto, Canada zichun.chong@mail.utoronto.ca

Zihang Huang

Yiran Li✉

Hans-Arno Jacobsen

University of Toronto Toronto, Canada zihang.huang@mail.utoronto.ca

University of Toronto Toronto, Canada one.li@utoronto.ca

University of Toronto Toronto, Canada jacobsen@eecg.toronto.edu

arXiv:2608.08968v1 [cs.SE] 10 Aug 2026

Abstract Microservice root cause analysis (RCA) requires correlating failures across heterogeneous telemetry within complex service dependency graphs. Existing methods often rely on a single telemetry modality; recent LLM-based approaches can suffer from unconstrained exploration and hallucination; and most systems stop at fault ranking without producing actionable incident response. We present GALA+, a graph-augmented LLM agentic framework centered on graph-guided investigation, which uses service dependencies to bound exploration and refine diagnosis through localized multi-modal evidence. For initial hypothesis generation, GALA+ combines complementary telemetry signals with STRIX, a novel trace- and graph-structure-aware scoring module. GALA+ then produces ranked diagnoses, incident summaries, and stratified action recommendations. We further introduce SURE-Score, a human-guided evaluation framework co-developed with industry SRE experts for assessing RCA-specific output quality beyond conventional text similarity metrics. On two microservice benchmarks, GALA+ consistently achieves the strongest overall results, surpassing the best LLM-based baseline by more than 25 percentage points in AC@1, while also receiving the highest ratings from both SURE-Score and independent human SRE evaluation.

CCS Concepts • Software and its engineering → Software maintenance tools; • Computing methodologies → Causal reasoning and diagnostics.

Keywords Root Cause Analysis, Multi-modal data, Large Language Model, Microservice System, Agentic Workflow ACM Reference Format: Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and HansArno Jacobsen. 2026. GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices. In Proceedings of the 41st

This work is licensed under a Creative Commons Attribution 4.0 International License. ASE ’26, Munich, Germany © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2882-2/2026/10 https://doi.org/10.1145/3832783.3834335

IEEE/ACM International Conference on Automated Software Engineering (ASE ’26), October 12–16, 2026, Munich, Germany. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/3832783.3834335

1

Introduction

Modern software systems increasingly adopt microservice architectures for scalability, maintainability, and deployment flexibility [14]. However, decomposing applications into numerous loosely coupled services communicating through intricate dependency graphs introduces significant operational complexity. When failures occur, root cause analysis (RCA) requires correlating symptoms across multiple services, analyzing heterogeneous telemetry data, and understanding complex inter-service dependencies—all under tight time pressure. Beyond identifying the faulty service, site reliability engineers (SREs) need a concise explanation of the failure’s propagation path and actionable steps to resolve the incident [71]. Consider a cloud-native application with six microservices 𝑆 1 –𝑆 6 and the dependency graph in Figure 1. In one incident, an alert is triggered on 𝑆 4 after its latency exceeds the acceptable threshold. An engineer’s initial inspection narrows the likely sources to 𝑆 4 itself, its upstream caller 𝑆 2 , and its downstream callee 𝑆 5 . However, the fault does not originate at the alerted service 𝑆 4 : while 𝑆 4 shows elevated latency, the underlying issue is a CPU-intensive slowdown in 𝑆 5 , whose delayed responses propagate upstream and inflate 𝑆 4 ’s latency. This example illustrates three core challenges. First, the service where the problem is observed may differ from the true fault origin, so RCA must reason over service dependencies rather than inspect services in isolation. Second, different telemetry signals may point to different suspects, so accurate diagnosis requires combining multiple modalities rather than relying on any single one. Third, identifying the faulty service alone is insufficient; engineers also need an explanation of how the failure propagated and what actions should be taken to mitigate it. In this work, we address all three challenges. However, existing approaches still fall short of addressing these three challenges in a unified manner. Statistical methods [30, 41, 63] address only challenge (2) partially: they operate on single modalities or use static heuristics that miss dynamic failure modes such as memory leaks triggered by specific API sequences [62]. Multi-modal approaches [17, 69, 75] fuse heterogeneous data but collapse it into unified representations, potentially losing the modality-specific signals that distinguish propagation victims from true root causes. Recent large language model (LLM) advancements have opened new

ASE ’26, October 12–16, 2026, Munich, Germany

Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and Hans-Arno Jacobsen

1. Service fault detected on S4

S1

S2 2. Most possible causes: S2, S4, S5

S3 S5

3. Investigate pod telemetry S4 4. Root cause S5 is found

S6

Service (S) Dependency Graph

Figure 1: Motivating example: a fault detected on 𝑆 4 is traced through the dependency graph to the true root cause 𝑆 5 .

possibilities for automating complex reasoning [6], with chain-ofthought [60], tree-of-thought [66], and agentic workflows such as ReAct [67] demonstrating potential for multi-step problem-solving. However, our empirical evaluation shows that unconstrained multi-agent reasoning may search across the entire service space without topological grounding, incurring excessive search overhead, hallucination, and hypothesis drift that ultimately degrades precision. Across all paradigms, most methods stop at ranking potential root causes, addressing neither challenge (3) nor the evaluation gap it creates: standard natural language generation (NLG) metrics (BLEU [38], ROUGE [31], BERTScore [72]) measure semantic overlap but cannot assess the causal grounding and operational specificity that SREs require. Effective LLM-based RCA therefore demands topology-constrained reasoning that bounds exploration to causally plausible paths, maintains cross-modal consistency, and produces not just fault rankings but also structured incident response. We present GALA+ (Graph-Augmented LLM Agentic Workflow), a framework that addresses these limitations by constraining agentic reasoning steps to the service dependency graph. To overcome single-modality blindness, GALA+ combines metrics-based causal ranking with STRIX, a novel scoring module that exploits trace patterns and dependency-graph topology, and consolidates their complementary signals into a unified candidate set. Each candidate’s raw telemetry is distilled into a compact diagnostic bundle preserving temporal patterns, dependency context, and error signals. To avoid the unfocused exploration that can arise in unconstrained agentic reasoning, we propose a graph-guided investigation mechanism in which parallel agents follow fault-propagation edges with confidence-driven, depth-bounded expansion, keeping the search localized to a small neighborhood around plausible root-cause candidates. The resulting evidence is synthesized into a final ranking, structured incident summaries, and stratified action recommendations. We further introduce SURE-Score, a human-guided LLM evaluation framework co-developed with industry SRE experts for RCA-specific output assessment. Evaluated on two microservice benchmarks, GALA+ achieves 74.44% AC@1 on OnlineBoutique and 73.33% on TrainTicket, outperforming the strongest baseline by over 25%, with the highest SURE-Score ratings across all dimensions confirmed by both dual-LLM grading and independent human SRE assessment. Our contributions are threefold: (1) Dual-Signal Root Cause Hypothesis Generation: We develop a hypothesis generation mechanism that fuses metrics-based initial ranking with STRIX, a novel trace- and graph-structure-aware scoring module, to produce complementary signals consolidated into a unified ranked candidate set.

(2) Graph-Guided Agentic Investigation: We introduce a topology-constrained parallel investigation framework in which agents follow fault-propagation edges with confidence-driven, depth-bounded expansion, substantially reducing the search space while preserving diagnostic completeness. (3) Incident Response Generation and Evaluation: We build a pipeline that produces ranked diagnoses, structured incident summaries, and stratified action recommendations. These outputs are evaluated with SURE-Score, a human-guided framework on which GALA+ achieves the highest scores across all four dimensions in both automated and human assessment.

2

Problem Statement

We consider microservice architectures [14] where a pod is the smallest deployable unit. Telemetry comprises three modalities: metrics (quantitative measurements such as CPU usage and latency [4]), logs (timestamped event records [68]), and traces (cross-service call relationships and timing). In distributed tracing, a span represents a single operation within a service, identified by a spanID and associated with a global traceID. Spans reference parent spans via parentSpanID, enabling construction of a service dependency graph: a directed acyclic graph (DAG) where nodes represent services and directed edges encode call relationships [51]. Let P denote the set of pods in a microservice deployment experiencing an incident. For each pod 𝑖 ∈ P, we observe metrics 𝑚𝑖 , logs 𝑙𝑖 , and traces 𝑡𝑖 collected during the incident window, with 𝑀, 𝐿, 𝑇 denoting the complete collections across all pods. A root cause is defined as the specific pod and failure mode (e.g., CPU hog in order-service) that, when addressed, resolves the incident. Given {𝑀,𝐿,𝑇 }, the RCA task requires producing: (1) Ranked Root Cause Identification: A prioritized ranking of candidate pods by fault likelihood. (2) Incident Summary: A human-readable narrative explaining the symptom-to-root-cause causal chain across all three telemetry modalities. (3) Prioritized Action Recommendations: Three remediation steps stratified across temporal horizons, covering immediate mitigation, permanent fix, and preventative measure.

3

The GALA+ Design

In this section, we present GALA+, a unified multi-modal framework for root-cause analysis in microservice-based systems. As shown in Figure 2, GALA+ operates in four coordinated phases that progressively transform raw telemetry into root cause insight and actionable remediation guidance. Phase I (Initial Hypothesis Generation) analyzes system-wide metrics and traces to produce a consolidated initial ranking of likely root-cause pods. Phase II (Pod-Centric Context Synthesis) distills each candidate pod’s metrics, logs, and traces into a compact context that preserves its temporal pattern, dependency neighborhood, and error signals. Phase III (Graph-Guided Agentic Reasoning and Re-ranking) dispatches investigator agents to examine high-ranked pods and traverse their service dependency graph, using localized evidence to iteratively refine the ranking. Phase IV (Final Output Preparation) consolidates these agentic evaluations into a final root-cause ranking and a structured incident summary with prioritized remediation recommendations.

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

III. Graph-Guided Agentic Investigation

I. Initial Hypothesis Generation Metrics-Based Ranking with BARO (RInitM) Trace-Based Ranking with STRIX (RInitT) Incident Report w. Metrics (M), Trace (C), and Log (L)

Consolidated Root Cause Ranking (RC)

Dependency Subgraph

Error-Log Distillation

IV. Diagnostic Synthesis and Incident Response

Global Evaluation Memory (.M )

Coarse Consolidation Rankings Agent

II. Pod-Centric Context Synthesis

Metrics Encoding

ASE ’26, October 12–16, 2026, Munich, Germany

Evidence Synthesis Agent Investigator Agent

Investigator Agent

Iterative Retrieval Diagnostic Bundle (Di)

Dependency Graph (GD)

Final Diagnostic Ranking (RFinal)

Remediation Agent

Final Root Cause Ranking

Incident Summarization

Site-Reliability Engineer (SRE)s

Action Recommendation

Figure 2: Overview of GALA+.

3.1

Initial Hypothesis Generation

Before dispatching LLM agents for deep pod-level investigation, GALA+ first requires a coarse estimate of which services are most likely responsible for the incident. In Phase I, this initial hypothesis is provided efficiently by analyzing metrics and traces with modality-specific methods, including our novel trace analysis technique STRIX. The resulting modality-level rankings are then integrated into a reliable preliminary root-cause estimate. Logs are excluded at this stage because many faults manifest silently without any error entries; their semantic content is instead analyzed in later phases. 3.1.1 Coarse Metrics-based Ranking. To generate an initial hypothesis from metrics, GALA+ instantiates this stage with BARO [41], which first identifies anomalous services via Bayesian online change-point detection and then constructs a causal DAG 𝐺 𝑀 over the collected metrics data 𝑀 to model inter-service dependencies and failure propagation paths. The resulting DAG is then used directly to score each candidate service by its causal influence on the Init =BARO(𝑀). observed anomalies, yielding an initial ranked list 𝑅𝑀 Nevertheless, metrics-based causal discovery methods, such as BARO, have two limitations reported in the literature [42]: they often misidentify causal edge directions, and inherently cannot distinguish genuine anomalies from routine operational fluctuations. To mitigate both shortcomings, we introduce a complementary trace-based ranking that directly examines dependency directions and performance stability. 3.1.2 Coarse Trace-based Ranking. As an orthogonal module to the metrics-based causal inference, we introduce STRIX (Structural Trace Ranking via Intermodal eXploration), which derives an initial root cause ranking from distributed traces. As a topology-driven method, STRIX analyzes the service dependency graph and inspects span-level latency patterns to surface a complementary hypothesis. Formally, given trace corpus 𝑇 , STRIX produces an initial ranked list 𝑅𝑇Init =STRIX(𝑇 ). Service Dependency Graph Construction. STRIX first constructs a weighted directed dependency graph 𝐺 D = (𝑉 ,𝐸) from the collected trace corpus 𝑇 , where 𝐷 denotes the service dependency. Each node 𝑣 ∈ 𝑉 represents a microservice, and each directed edge (𝑢,𝑣) ∈ 𝐸 represents the aggregated invocation relationship from service 𝑢 to

service 𝑣, consolidated across all observed spans in𝑇 . Edge and node attributes are populated by aggregating span-level measurements across all matching records, including latency statistics (mean, P50, P95, P99, max, standard deviation, and total cumulative latency) and call volume (i.e., the total number of spans observed on that edge or node). These attributes form the basis for the three diagnostic dimensions computed in the next step. Three Diagnostic Dimensions. Based on this service dependency graph, STRIX evaluates three diagnostic dimensions for each service node 𝑣, integrating both the graph’s topology and its aggregated attribute statistics. • Instability (𝜙 1 ) captures the hallmark of resource contention and transient faults: a service under stress develops a pronounced latency tail while its median remains comparatively stable [51]. This separates true bottlenecks exhibiting sharp latency spikes from services that are simply slow but predictable. We define 𝑝 (𝑣) 𝜙 1 (𝑣) = 𝑝 99 , the ratio of tail latency (𝑝 99 ) of incoming calls to 50 (𝑣) the median (𝑝 50 ) as the Instability score for service 𝑣. • Centrality (𝜙 2 ) captures the structural tendency that services with high fan-in and strong graph centrality are more likely to be true root causes, whereas high fan-out services are often fault-propagation intermediaries [61]. We formalize this by weighting PageRank with the fan-in to fan-out ratio: 𝜙 2 (𝑣) =PageRank(𝑣) ·

1+in-degree(𝑣) . 1+out-degree(𝑣)

This jointly measures global structural importance via PageRank and local dependency pattern via the degree ratio, elevating services that are widely reachable and heavily depended upon. • Impact (𝜙 3 ) captures how much delay originates from a service’s own processing rather than from waiting on downstream callees, a decomposition established in distributed trace analysis [29]. A service whose average latency substantially exceeds that of its slowest callee is more likely to be the true bottleneck. Let AvgLatency(𝑣) denote the mean span duration of service 𝑣 across all observed calls, and TotalLatency(𝑣) its cumulative span duration aggregated over all spans. We define   max (𝑣,𝑢 ) ∈𝐸 AvgLatency(𝑢) 𝜙 3 (𝑣) =TotalLatency(𝑣) ·max 0,1− , AvgLatency(𝑣)

ASE ’26, October 12–16, 2026, Munich, Germany

where the bracketed term estimates the fraction of 𝑣’s average latency attributable to its own processing by discounting the maximum average latency of its outgoing callees, and TotalLatency(𝑣) scales this local inefficiency by the volume of affected traffic to produce a globally comparable impact score. Rank Aggregation via Reciprocal Rank Fusion. Since 𝜙 1 , 𝜙 2 , and 𝜙 3 span fundamentally different numerical ranges, direct weighted combination would require domain-specific calibration. STRIX instead applies Reciprocal Rank Fusion (RRF) [11], which operates on ordinal rank positions rather than raw scores and is thus scale-invariant by construction. Each service 𝑣 receives an RRF score Í 1 aggregated across all three dimensions: RRF(𝑣) = 3𝑗=1 𝜖+rank(𝑣,𝜙 𝑗) where 𝜖 = 60 is a smoothing constant following the original implementation [11]. With RRF, services that rank consistently high across all three dimensions are naturally elevated, while those that dominate in only one are moderated. The final output of STRIX is the ranked list 𝑅𝑇Init = STRIX(𝑇 ), defined as the services ordered by their RRF scores in descending order. This provides a robust, topology-grounded initial hypothesis Init , that is structurally complementary to the time-series-driven 𝑅𝑀 and the two are subsequently consolidated in Section 3.1.3. 3.1.3 Consolidation Agent and Hypothesis Fusion. The two coarse rankings above carry complementary but asymmetric information. Init provides service-fault-type pairs grounded in causal structure 𝑅𝑀 learning, while 𝑅𝑇Init provides topology-driven service orderings without fault-type attribution. A candidate appearing only in 𝑅𝑇Init (hereafter trace-only) may be a structurally prominent bottleneck whose fault type is unknown, whereas a candidate appearing Init (metrics-only) may carry a precise fault label but lack only in 𝑅𝑀 corroborating structural evidence. Naive score-based merging would ignore this asymmetry, so we design a Consolidation Agent that reasons explicitly over each case to produce a unified ranking. Init and 𝑅 Init as input, the agent is prompted to: (1) identify Given 𝑅𝑀 𝑇 cross-modal consensus candidates that appear prominently in both rankings and assign them elevated priority; (2) infer plausible fault types for trace-only candidates by leveraging fault-type Init ; and (3) retain metrics-only candidates with context from 𝑅𝑀 appropriately reduced confidence when trace evidence is absent. Each candidate in the output is assigned a consolidated confidence score 𝑐ˆ𝑖 ∈ [0, 1] with explicit evidence provenance drawn from one or both modalities, reflecting the strength of cross-modal corroboration: candidates confirmed by both rankings receive the highest 𝑐ˆ𝑖 , trace-only candidates receive moderate scores with inferred fault types, and metrics-only candidates receive the lowest. This provenance-aware scoring ensures that downstream agents can distinguish well-substantiated hypotheses from speculative ones. The result is a fused ranking 𝑅 C ={(𝑣𝑖 ,ˆ𝑐𝑖 )} whose top-𝑘 candidates, each paired with its consolidated confidence, are forwarded to the graph-guided investigation in Section 3.3.

3.2

Pod-Centric Context Synthesis

Raw metrics, logs, and traces are too voluminous and heterogeneous to be passed directly to an LLM. Therefore, in Phase II, each candidate pod’s telemetry is condensed into a compact diagnostic

Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and Hans-Arno Jacobsen

bundle 𝐷𝑖 that preserves the key temporal patterns, dependency context, and error signals needed for targeted pod-level analysis. For each pod 𝑖 and each monitoring metric, we serialize persecond measurements within the incident window [𝜏 −Δ𝜏,𝜏 +Δ𝜏] into a timestamped JSON dictionary m𝑖∗ , where 𝜏 is the anomaly onset timestamp derived from the incident alert and Δ𝜏 is a fixed look-around window [71]. This representation replaces an earlier Base64-encoded line-chart design; empirically, JSON reduces token consumption while improving diagnostic accuracy. For structural context, we extract a 1-hop subgraph 𝑔𝑖 from the service dependency graph, comprising 𝑣𝑖 , its direct predecessors and successors, and associated edge attributes such as call volume and latency statistics. For log evidence, a LogDistiller processes the raw stream 𝑙𝑖 by retaining only error-level and exception entries, ordered by timestamp and deduplicated at the message level. If the retained entries exceed a token budget, we keep a representative sample; otherwise, we supplement them with a small number of non-error entries to distinguish silent failures from log absence, yielding 𝑙𝑖∗ . These three artifacts are then assembled into the diagnostic bundle 𝐷𝑖 ={m𝑖∗,𝑔𝑖 ,𝑙𝑖∗ } using a structured text template with three labeled sections: m𝑖∗ as JSON,𝑔𝑖 as an annotated adjacency list, and𝑙𝑖∗ as a chronologically ordered message list. This format preserves each modality in its native representation within a single token-efficient context.

3.3

Graph-Guided Agentic Investigation

Given the initial ranking 𝑅 C , Phase III performs a graph-guided investigation that re-examines highly ranked hypothesis candidates through localized multi-modal reasoning and extends to neighboring services when local evidence is insufficient. The key idea is to couple LLM-based agentic analysis with the structure of the service dependency graph: each Investigator Agent starts from a candidate in 𝑅 C , reasons over its diagnostic bundle 𝐷𝑖 , and, when the evidence is inconclusive, examines services in its local dependency neighborhood. The structured assessments produced along these investigation branches are accumulated in a shared investigation memory M for downstream synthesis. In this way, GALA+ deepens the investigation beyond the initial ranking while keeping exploration confined to a small, structurally meaningful neighborhood around plausible root-cause candidates. The Graph-Guided Investigation Algorithm (Algorithm 1) formalizes this process. Starting from the top-𝑘 entries of 𝑅 C , GALA+ invokes DeepDive on each candidate. For a service 𝑣, the Investigator Agent reasons over the diagnostic bundle 𝐷 𝑣 together with the consolidated ranking 𝑅 C , and produces a structured assessment consisting of a root-cause confidence 𝑐ˆ𝑣 ∈ [0, 1], an evidence summary 𝜎𝑣evid , a causal explanation 𝜎𝑣causal , and supporting and contradicting factors 𝐹 𝑣+ and 𝐹 𝑣− . Every assessment is committed to the shared memory M as soon as the agent returns, so that M retains a record of every investigated service. The resulting confidence determines whether the investigation stops or extends. If 𝑐ˆ𝑣 ≥𝜃 , the candidate’s local evidence is considered sufficient and the branch terminates. Otherwise, GALA+ examines the candidate’s dependency neighbors in randomized batches of size 𝑏, stopping as soon as a batch contains a high-confidence service or all neighbors have been examined.

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

Algorithm 1: Graph-Guided Investigation Input: Consolidated ranking 𝑅 C , service dependency graph 𝐺=(𝑉 ,𝐸 ), diagnostic bundles {𝐷𝑖 }, parallel branches 𝑘, threshold 𝜃 , batch size 𝑏 Output: Global evaluation memory M 1 M ← ∅; C 2 for each (𝑣𝑖 ,𝑐ˆ𝑖 ) ∈ TopK(𝑅 , 𝑘 ) in parallel do 3 DeepDive (𝑣𝑖 ); Function DeepDive(𝑣): 𝑐ˆ𝑣 ,𝜎𝑣evid ,𝜎𝑣causal ,𝐹 𝑣+ ,𝐹 𝑣− ← Investigator(𝐷 𝑣 , 𝑅 C ); 6 M ← M ∪ { (𝑣, 𝑐ˆ𝑣 , 𝜎𝑣evid , 𝜎𝑣causal , 𝐹 𝑣+ , 𝐹 𝑣− ) }; 7 if 𝑐ˆ𝑣 ≥ 𝜃 then 8 return ; // Sufficient local evidence

4

5

9 10 11 12 13 14 15 16 17 18

N (𝑣) ← Predecessors(𝑣,𝐺 ) ∪ Successors(𝑣,𝐺 ); U ← N (𝑣); while U ≠ ∅ do B ← RandomSample( U, min{𝑏,| U | } ); U ← U \ B; for each 𝑢 ∈ B do 𝑐ˆ𝑢 ,𝜎𝑢evid ,𝜎𝑢causal ,𝐹𝑢+ ,𝐹𝑢− ← Investigator(𝐷𝑢 , 𝑅 C ); M ← M ∪ { (𝑢, 𝑐ˆ𝑢 , 𝜎𝑢evid ,𝜎𝑢causal , 𝐹𝑢+ , 𝐹𝑢− ) }; if max𝑢 ∈B 𝑐ˆ𝑢 ≥ 𝜃 then return ;

// High-confidence neighbor

The investigation is bounded by the size of each candidate’s local neighborhood. A candidate with sufficient local evidence requires one assessment; otherwise, at most |N (𝑣)| additional services are examined. Across the top-𝑘 candidates, the worst-case work is 𝑂 (𝑘Δ), where Δ is the maximum degree of 𝐺, while the 𝑘 branches execute concurrently. This design gives GALA+ a bounded search procedure that adapts to the strength of local evidence. Rather than relying solely on the initial ranking or allowing unconstrained agentic exploration, it performs additional reasoning only when a candidate’s local evidence is inconclusive, and restricts that reasoning to structurally related services. After all investigations terminate, M contains the structured assessments for the explored services, which are then passed to the Evidence Synthesis Agent in Phase IV for final ranking and incident-report generation.

3.4

anchoring treats the initial confidence scores from 𝑅 C as grounded priors, preventing hallucination-driven re-ranking in which a locally persuasive assessment overrides stronger system-level evidence. Second, chain coherence requires that the top-ranked candidate’s causal reasoning chain 𝜎𝑣causal explain the downstream symptoms observed in other explored branches through their evidence summaries 𝜎𝑢evid and contradicting factors 𝐹𝑢− . A candidate is promoted to rank 1 only if it not only appears locally plausible, but also serves as the most coherent global explanation of the evidence distributed across M. This synthesis step is critical for distinguishing true root causes from downstream victims that may exhibit strong local symptoms. Remediation Consolidation. Building on 𝑅 Final and its associated reasoning traces, the Remediation Agent generates an incident report that couples diagnosis with evidence-grounded action recommendations. Specifically, it takes as input the final ranking, the supporting rationale, and the inferred causal chain of failure propagation, and produces a structured incident summary together with three remediation actions stratified across temporal horizons: an immediate mitigation to restore service stability, a permanent fix that addresses the underlying failure mechanism, and a preventative measure tailored to the observed propagation pattern and dependency topology traversed in Phase III. The resulting recommendations are incident-specific, traceable to the synthesized evidence, and directly actionable for SREs. Taken together, Phase IV produces a final root-cause ranking, a structured incident summary, and prioritized remediation recommendations spanning immediate mitigation, permanent fix, and future prevention. By converting investigation findings into a coherent diagnosis and actionable response plan, GALA+ helps SREs move from root-cause identification to incident resolution more quickly and with stronger evidential support.

4

Human-Guided Evaluation of RCA Reports

(A) Checklist Construction

Think-Aloud Elaboration LLM-based checklist Merging Attributes

LLMs (Gemini, Llama, Claude, GPT) Four RCA report evaluation dimensions

Diagnostic Synthesis and Incident Response

In Phase IV, branch-specific findings collected during graph-guided investigation are transformed into a single coherent diagnosis and an evidence-grounded incident response. Its key role is not merely to summarize the contents of investigation memory M, but to reconcile potentially competing agent assessments, preserve consistency with the initial system-level signals, and convert the final diagnosis into remediation guidance that is directly tied to the inferred failure mechanism. Through this design, GALA+ bridges the gap between root-cause identification and operational incident handling. Evidence Synthesis. Once all Investigator Agents terminate, the Evidence Synthesis Agent synthesizes the contents of M into a final diagnostic ranking 𝑅 Final . Rather than relying on LLM reasoning alone, this synthesis is governed by two arbitration rules. First, signal

ASE ’26, October 12–16, 2026, Munich, Germany

SRE experts

(B) Evaluation Pipeline

Human-based checklist Final checklist per dimension

Report grading per dimension RCA report per incident

Evaluator LLM

LLM-based Consolidation

Q1. Q2. Q3. ... Qn.

Score aggregation

SURE-score (from 1 to 5)

Figure 3: SURE-Score framework. Evaluating RCA-generated summaries and remediation recommendations should focus on whether the outputs are logically coherent, causally grounded in the observed failure, and operationally useful for incident response. However, standard natural language generation metrics such as BLEU [38], ROUGE [31], and

ASE ’26, October 12–16, 2026, Munich, Germany

BERTScore [72] primarily capture lexical or semantic overlap and therefore fail to assess these RCA-specific dimensions. In contrast, LLM-based evaluation frameworks that incorporate human guidance and feedback can provide more reliable and contextually valid assessments than such automated metrics [10]. To address this gap, we introduce SURE-Score (SUmmarization REcommendation Score), a novel human-guided evaluation framework for RCA-specific textual outputs, organized into two steps (Figure 3). Step A: Checklist Construction. SURE-Score adopts the ThinkAloud Elaboration protocol from InteracEval [10], which provides a mechanism for both human experts and LLMs to externalize their reasoning about what constitutes a high-quality RCA report. Together with domain experts at Company A, we first co-develop four RCA-critical evaluation dimensions: (1) Evidence Groundedness, which requires every reasoning step in the narrative to be explicitly supported by concrete diagnostic evidence, such as a log entry, metric deviation, or trace anomaly, while penalizing hallucinations, unsupported claims, and steps that cannot be traced to observable artifacts. (2) Operational Feasibility, which evaluates whether the proposed remediation actions are concrete and implementable within the system architecture, severely penalizing vague directives such as “improve monitoring”. (3) Diagnostic Precision, which requires specific service names, pod identifiers, and quantitative impact metrics (e.g., "latency increased from 200ms to 1.5s") rather than generic placeholder descriptions. (4) Triage Logic, which evaluates whether the summary follows a professional triage flow from symptoms to pivot evidence to root cause, with critical findings presented first and a coherent reasoning pathway synthesized for the SREs. Eight SRE experts from Company A, with four to ten years of operations experience, and four LLMs (GPT-4.1-mini [34], Gemini-2.5Pro [19], Claude-Sonnet-4.5 [2], and Llama-4-Maverick [33]) then individually perform Think-Aloud elaboration for each dimension, generating candidate attributes that reflect their understanding of what strong RCA performance entails. We use Qwen3 [44] to merge and deduplicate the collected attribute lists into a final, dimension-specific binary checklist of Yes/No criteria. This multi-source construction ensures that the checklist integrates practitioners’ operational intuitions with LLM-derived perspectives, yielding an evaluation instrument that is both practically grounded and systematically structured. Step B: Evaluation Pipeline. We first construct a validated reference report for each failure case: we prompt an LLM with the ground-truth faulty pod, failure mode, and fault onset time, together with a description of the microservice system and SRE-authored few-shot examples, and constrain the output to a fixed template comprising an incident summary and three stratified remediation recommendations. SRE experts then manually verify each reference, confirming that the diagnosis is consistent with the ground-truth fault type and that the recommended actions are operationally plausible in the target system. For each RCA report, Qwen3 [44] and Llama-4-Maverick [33] independently evaluate the report against the constructed checklist, using the validated reference as ground truth. Each binary criterion is assessed separately, and the positive responses are aggregated within each dimension and rescaled to a

Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and Hans-Arno Jacobsen

1–5 scale, yielding an overall SURE-Score from each evaluator. To reduce self-preference bias [9], these evaluator models are chosen to be different from the LLM used to generate the report under assessment. This cross-model grading design, together with the human-validated checklist, mitigates the risk that an evaluator favors its own generation style. We adopt this framework following InteracEval, which has been empirically validated to align well with human judgments [10].

5

Evaluation

We evaluate GALA+ by addressing the following research questions: • RQ1: How effective is GALA+ in root cause ranking compared to state-of-the-art methods? • RQ2: How effective is GALA+ in incident summarization and action recommendation? • RQ3: How does each component of GALA+ affect overall RCA performance? • RQ4: How does the choice of LLM affect the performance, efficiency, and cost of GALA+? • RQ5: How do parameters in agentic investigation (Phase III) affect GALA+’s performance?

5.1

Experiment Setup

Microservice Applications. We evaluate GALA+ on two microservice benchmark datasets. The OnlineBoutique (OB) dataset comprises 90 injected-fault scenarios across five services of a 17service e-commerce application communicating via RPC, spanning six fault types (CPU hog, memory leak, disk I/O stress, network delay, socket errors, and packet loss) with three independent repetitions each [43]. The TrainTicket (TT) dataset is a larger and more complex benchmark based on a ticket-booking system with 69 microservices, also comprising 90 fault scenarios across the same six fault types [43]. Together, these two datasets provide complementary evaluation environments: OB tests precision on a compact, well-connected service graph, while TT stresses scalability and reasoning depth on a significantly larger topology. We additionally provide results on the AegisLab benchmark [16], which spans 25 distinct fault types, using 100 sampled fault cases. This additional evaluation compares GALA+ against a representative subset of baselines; detailed results are reported in our technical report [56], with the main findings summarized in Section 5.3.1. LLM Settings. We evaluate GALA+ with seven LLMs spanning both proprietary and open-source families. Proprietary models are accessed via their official APIs: gemini-2.5-flash [18], claudesonnet-4-5 [2], GPT-4.1-mini [34], and GPT-4.1 [35]. Open-source models—deepseek-v3.1 [12], Qwen3-Next-80B-A3B-Instruct [44], and llama-4-maverick [33]—are served via the Together AI inference platform [57]. The selection of both large and small models is informed by Company A’s real-world AIOps deployment experience: in production-grade incident management pipelines, thousands of incidents are analyzed daily, making cost-effectiveness a first-class requirement alongside diagnostic accuracy. Accordingly, we designate GPT-4.1-mini as the default backbone for all reported GALA+ results, as it delivers near-frontier accuracy at a fraction of the cost of larger models (Table 5). All models are invoked with temperature 1.0.

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

Parameter Settings. For the Graph-Guided Investigation Algorithm, the default settings in GALA+ are: parallel branches 𝑘=6, pruning threshold 𝜃 =0.6, batch size 𝑏=2. Baselines. We compare GALA+ against two categories of baselines. Non-LLM baselines: Granger [30], CausalRCA [63], and BARO [41], representing RCA paradigms based on temporal dependency analysis, structured causal modeling, and graph-centrality reasoning, respectively. We also report results for PC [52], LiNGAM [50], TraceRCA [29], CIRCA [26], RCD [21], HeMiRCA [76], and Nezha [69]. For methods requiring causal graphs for root-cause ranking, we distinguish PageRank-based [5] (-PR) and random-walk-based [24] (-RW) variants with corresponding suffixes. LLM-based baselines: We include mABC [73], Flow-of-Action [39], RCA-Agent from OpenRCA [65], and GALA [55], a previous version of this work that differs from the current GALA+ in several design components. These represent recent multi-agent and agentic LLM approaches to RCA. We further implement two additional LLMbased baselines to broaden the comparison. For GraphRAG [15], we augment the service dependency graph 𝐺 D with nodes representing service metrics and logs. For ReAct [67], we implement an agent that iteratively reasons over telemetry with a fixed action space covering service inspection, neighbor traversal, metric and log queries, and final ranking. All baselines except Nezha have released source code; we reimplemented Nezha on a best-effort basis. Environment Details. All experiments were executed on an Ubuntu 22.04 virtual machine with eight physical CPU cores, 16 GB RAM, and 500 GB SSD storage, using Python 3.10.12 without GPU.

5.2

Evaluation Metrics

Ranking metrics (RQ1, RQ3–5). Following standard practice in the RCA literature [30, 41–43, 61, 69], we report three ranking metrics: AC@1, the proportion of incidents for which the top-ranked candidate is the true root cause; AC@3, the proportion of incidents for which the true root cause appears in the top three; and MRR, the mean reciprocal rank of the true root cause across all incidents. All three metrics are computed at the pod level: a ranked candidate counts as correct when it matches the ground-truth faulty pod, and no fault-type match is required. Text quality metrics (RQ2). We measure GALA+’s summarization and recommendation quality using BERTScore [72] (BE), cosine similarity [47] (COS), ROUGE-L [31] (RL), and METEOR [3] (MT) as traditional NLG baselines, and our SURE-Score (Section 4) as the primary evaluation framework.

5.3

Evaluation Results

5.3.1 RQ1: Overall Root Cause Ranking Effectiveness. Table 1 reports accuracy, runtime, and total token consumption for all methods on OB and TT. GALA+ achieves the highest performance across all ranking metrics on both datasets, attaining 74.44% AC@1 and 98.89% AC@3 on OB, and 73.33% AC@1 and 85.56% AC@3 on TT. Comparison with Non-LLM Baselines. The strongest non-LLM baseline differs by system: CIRCA leads on OB (66.67% AC@1, 90.00% AC@3, and 0.789 MRR), whereas BARO leads on TT (66.67% AC@1, 82.22% AC@3, and 0.759 MRR). GALA+ outperforms the best

ASE ’26, October 12–16, 2026, Munich, Germany

non-LLM method on each system, by +7.77 and +6.66 AC@1 percentage points on OB and TT, respectively. On OB, BARO frequently overpredicts Redis because its disk I/O exhibits persistent spikes across many incidents, causing this spurious signal to dominate the ranking. This failure mode highlights the risk of relying on a single telemetry modality for RCA. CausalRCA times out entirely on TT (over one hour per case), highlighting the scalability limitations of causal structure learning on larger service graphs. Comparison with LLM-based Baselines. Flow-of-Action is the strongest prior LLM-based approach (48.35% AC@1 on OB; 47.78% on TT), yet GALA+ surpasses it by +26.09 and +25.55 percentage points in AC@1, respectively, showing the advantage of graph-guided investigation over fixed procedural flows. GALA is outperformed by GALA+ because it refines its root cause hypothesis from a single retrieved telemetry bundle for one pod at a time, which often results in insufficient evidence gathering and validation. mABC performs poorly on both datasets because its blockchain-inspired collaboration protocol frequently hallucinates non-existent service names. GraphRAG consumes the most tokens among all methods (78.2K on OB; 149.3K on TT) yet achieves only 21.35% and 7.78% AC@1, confirming that graph-based retrieval without iterative multi-modal reasoning is insufficient for precise fault localization. ReAct improves over GraphRAG while using far fewer tokens, but its unconstrained action space often causes the investigation to drift away from plausible root-cause candidates, especially on TT’s larger service graph. Time and Token Efficiency. GALA+ requires about one minute per incident on average, comparable to other LLM-based methods such as RCA-Agent and Flow-of-Action. Although non-LLM methods such as BARO are faster, GALA+ is substantially more robust in identifying the true root cause and additionally produces rich, actionable textual outputs, making the added runtime acceptable in practice. Notably, CIRCA, RCD, PC, and Granger are faster than GALA+ on OB but scale poorly on TT, becoming much slower as the dependency graph expands. GALA+ consumes 32.7K total tokens on OB and 90.4K on TT, reflecting the cost of exploring multiple candidate branches and retaining reasoning traces for final report generation. Even so, it uses roughly half the tokens of RCA-Agent on OB (32.7K vs. 60.0K) while achieving 3.7× higher AC@1 and lower runtime (52.87s vs. 89.81s), as graph-guided pruning avoids exhaustive tool-call iteration. Flowof-Action uses fewer tokens, but at the cost of roughly 26 percentage points in AC@1. Overall, GALA+’s token overhead is a deliberate trade-off for substantially stronger diagnostic performance. Statistical Significance. We apply McNemar’s test [40] on perincident majority-vote outcomes across three runs to assess whether GALA+’s improvement over Flow-of-Action is statistically significant. On OB, GALA+ correctly diagnoses 27 incidents that Flow-ofAction misses versus 4 in the reverse direction; on TT, 35 vs. 12. Both asymmetries are significant (𝑝 < 0.001, one-sided exact binomial). AegisLab Results. AegisLab covers 25 distinct fault types, compared with six in OB and TT. On 100 sampled cases, GALA+ achieves the best MRR, AC@1, and AC@3 among the evaluated methods. It improves MRR over the strongest baseline, Flow-of-Action, from 0.38 to 0.48 (+26% relative, comparable to its +28% relative margin on TT), and AC@3 from 0.52 to 0.62. The detailed comparison is reported in our technical report [56].

ASE ’26, October 12–16, 2026, Munich, Germany

Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and Hans-Arno Jacobsen

Table 1: RQ1 results on Online Boutique (OB) and Train Ticket (TT). Best in bold, second best underlined. Improvement = relative ranking accuracy improvement over second best in column. Online Boutique Type

Method

Non-LLM

LLM-based

AC@3 (%)

MRR

Time (s)

Tokentotal

AC@1 (%)

AC@3 (%)

MRR

Time (s)

Tokentotal

BARO CausalRCA PC-PR PC-RW LiNGAM-PR LiNGAM-RW Granger-PR Granger-RW TraceRCA Nezha CIRCA HeMiRCA RCD

14.44 26.67 15.56 3.33 0.00 3.33 12.22 3.33 13.19 28.89 66.67 37.78 64.44

87.78 68.89 36.67 43.33 1.11 43.33 35.56 44.44 61.54 52.22 90.00 58.89 72.22

0.524 0.490 0.332 0.290 0.086 0.292 0.313 0.293 0.340 0.436 0.789 0.503 0.713

0.05 1,575 6.68 6.57 0.50 0.69 10.86 11.15 3.23 76.21 5.10 0.10 51.90

-

66.67 1.11 1.11 0.00 1.11 3.33 1.11 64.44 15.56 53.33 21.11 10.00

82.22 6.67 4.44 0.00 4.44 6.67 3.33 75.56 40.00 71.11 34.44 18.89

0.759 0.082 0.077 0.021 0.079 0.087 0.076 0.711 0.324 0.636 0.322 0.151

0.27 T/O 1,746 1,799 48.90 60.53 400 406 13.45 513 2,164 0.40 164.9

-

GraphRAG ReAct

21.35 44.44

35.96 65.56

0.331 0.549

9.58 14.73

78.2K 12.6K

7.78 24.44

14.44 41.39

0.163 0.353

19.62 15.92

149.3K 28.3K

mABC RCA-Agent GALA Flow-of-Action

13.33 20.00 38.89 48.35

44.44 48.89 86.67 70.33

0.296 0.337 0.639 0.590

57.02 50.63 49.48 36.50

21.9K 60.0K 13.8K 9.6K

12.22 32.22 41.94 47.78

23.33 58.89 72.04 74.44

0.196 0.440 0.602 0.627

92.98 89.81 136.33 123

38.9K 116.4K 37.6K 24.5K

GALA+ Improvement

74.44 +12%

98.89 +10%

0.854 +8%

52.87 -

32.7K -

73.33 +10%

85.56 +4%

0.801 +6%

63.16 -

90.4K -

Table 2: RCA text report evaluation. Traditional metrics: BE = BERTScore, CS = Cosine Similarity, RL = ROUGE-L, MT = METEOR. SURE-Score results with two LLMs (Q = Qwen3, L = Llama-4), E = Evidence Groundedness, O = Operational Feasibility, D = Diagnostic Precision, T = Triage Logic, F = Final (mean). Best in bold.

E

SURE-Score (L) O D T F

OB

Trad. SURE-Score (Q) BE CS RL MT E O D T F

BARO+LLM -.10 .28 .13 .16 1.0 1.4 1.7 1.3 1.3 1.0 3.4 2.2 2.2 2.2 mABC -.23 .21 .10 .15 1.0 1.0 1.8 1.4 1.3 1.3 1.0 2.6 3.7 2.2 Flow-of-A. -.23 .19 .11 .14 1.4 1.0 2.3 1.0 1.4 1.7 1.0 2.4 1.2 1.6 RCA-Agent -.44 .08 .09 .01 1.0 1.0 1.6 1.0 1.2 1.0 1.0 1.2 1.0 1.0 GALA+ .04 .35 .13 .21 1.8 3.5 4.4 3.3 3.3 5.0 3.7 4.5 5.0 4.5

TT

Method

Train Ticket

AC@1 (%)

BARO+LLM -.06 .37 .17 .15 1.0 1.4 2.1 1.4 1.5 1.0 3.1 2.5 1.9 2.1 mABC -.17 .25 .11 .16 1.0 1.0 2.1 1.4 1.4 1.2 1.0 2.8 3.1 2.0 Flow-of-A. -.25 .23 .13 .14 1.3 1.0 2.8 1.0 1.5 2.3 1.0 2.8 1.2 1.8 RCA-Agent -.41 .14 .13 .01 1.0 1.0 1.8 1.0 1.2 1.0 1.0 1.2 1.0 1.1 GALA+ .06 .38 .13 .21 1.4 3.3 4.2 3.2 3.0 4.9 3.5 4.4 4.8 4.4

GALA+ achieves the strongest root-cause ranking performance on all datasets, and its improvement over the best prior LLM-based baseline is statistically significant. Although GALA+ incurs moderate runtime and token overhead, this cost is offset by substantially higher diagnostic accuracy and actionable output generation. 5.3.2 RQ2: Incident Summarization and Action Recommendation. Table 2 reports SURE-Score results evaluated by two independent LLMs from different model families—Qwen3 and Llama-4-Maverick— alongside traditional natural language generation metrics. Since BARO produces only fault rankings, we evaluate BARO+LLM, which uses the same backbone (GPT-4.1-mini) as GALA+ for fair comparison.

Traditional Metrics are insufficient. GALA+ attains the best BERTScore, cosine similarity, and METEOR on both OB and TT, yet all four metrics are weakly discriminative: BERTScore is mostly negative, while cosine similarity and ROUGE-L vary little across methods. For example, on TT, BARO+LLM attains nearly the same cosine similarity as GALA+ (0.37 vs. 0.38) but a much lower SURE-Score (2.1 vs. 4.4), showing that surface-level text similarity fails to capture the causal grounding and operational specificity required for RCA reports. SURE-Score results. Across both evaluator LLMs, GALA+ consistently achieves the highest overall SURE-Score on both OB and TT, and ranks first on all evaluation dimensions. Under Qwen3, GALA+ scores 3.3 on OB and 3.0 on TT; under Llama-4-Maverick, it scores 4.5 on OB and 4.4 on TT. Although the absolute scores differ and Llama-4-Maverick tends to give a higher score, the two evaluators agree on the central result that GALA+ clearly outperforms all baselines, with only minor variation in the ranking among baselines. This overall consistency in ranking suggests that SURE-Score captures robust quality differences rather than evaluator-specific artifacts. The largest gains appear in Operational Feasibility and Diagnostic Precision, reflecting GALA+ ’s ability to translate accurate diagnoses into specific, implementable recommendations. Evidence Groundedness Diverges Across Evaluators. This dimension shows the greatest divergence between evaluators. Under Qwen3, all methods receive scores close to 1.0, including GALA+ (1.8 on OB), whereas under Llama-4-Maverick, GALA+ reaches 5.0 while the baselines remain low. This contrast suggests that the two evaluators apply different standards for explicit evidential support: Qwen3 favors verbatim citation of logs or metric values, whereas Llama-4-Maverick also credits paraphrased textual references. Despite this difference in grading strictness, GALA+ ranks highest

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

ASE ’26, October 12–16, 2026, Munich, Germany

Table 3: Human SRE evaluation. E/O/D/T/F as in Table 2. D

T

F

BARO+LLM Flow-of-Action GALA+

2.3 3.1 4.2

2.1 1.4 4.4

2.3 3.4 4.6

2.8 2.4 4.5

2.4 2.6 4.4

BARO+LLM Flow-of-Action GALA+

2.3 2.8 3.8

2.0 1.5 4.6

2.4 3.3 4.4

2.8 2.5 4.0

2.4 2.6 4.2

under both evaluators, indicating that its multi-modal reasoning consistently yields more evidence-grounded narratives. Human Evaluation. To validate that SURE-Score aligns with practitioner judgment, three SREs with production incident response experience independently graded 540 reports (90 incidents × 3 methods × 2 datasets) on the same four dimensions using the same 1–5 scale. Table 3 reports the averaged results. GALA+ outperforms Flow-of-Action across all dimensions, with the largest gap on Operational Feasibility (+3.0 average): Flow-of-Action produces no remediation recommendations and scores near the floor (1.4–1.5), while GALA+’s three-tier stratified recommendations consistently exceed 4.0. The BARO+LLM’s overall performance is low because its narratives do not cite any concrete telemetry. The evaluators independently identified Evidence Groundedness as a weak dimension for all methods, confirming that LLMs paraphrase rather than cite telemetry and validating that SURE-Score captures a genuine quality gap. Human scores (2.4–4.4) are directionally consistent with automated SURE-Score ratings, though higher in absolute terms because evaluators give partial credit for implicit evidence references. This offset does not affect method ranking: both automated and human evaluations agree that GALA+ dominates on every dimension. Effect of Report Length. While SURE-Score does not normalize for report length, it is not driven by length either: the checklist requires exact service and pod identifiers, quantitative impact figures, evidence-anchored reasoning steps, and implementable remediation actions, rather than generic long-form descriptions. On OB, mABC produces longer reports than GALA+ on average (339 vs. 322 tokens) yet is graded 2.2 against GALA+’s 4.5 by Llama-4-Maverick and 1.3 against 3.3 by Qwen3. This suggests that SURE-Score is driven more by diagnostic precision and operational usefulness than by length alone. Evaluator Agreement. To quantify SURE-Score reliability, we assess two forms of agreement. First, pooled inter-evaluator agreement between Qwen3 and Llama-4-Maverick across all methods, dimensions, and datasets yields Kendall’s 𝜏 = 0.68 [49] and Spearman’s 𝜌 =0.84 [48] (both 𝑝 < 0.001, 𝑛 =50), confirming that the two independent evaluator LLMs produce consistent method rankings despite differing in absolute scoring scales. Second, Spearman correlation between averaged automated SURE-Score and the independent human SRE ratings (Table 3) is 𝜌 =0.92 (𝑝 < 0.001, 𝑛 =20), indicating that SURE-Score serves as a reliable proxy for expert assessment. Traditional NLG metrics fail to differentiate RCA report quality. GALA+ outperforms all baselines under SURE-Score, confirmed by cross-model evaluator agreement and independent human SRE evaluation. Evidence Groundedness remains an open challenge. 5.3.3 RQ3: Ablation Study. Figure 4 presents an ablation study isolating the contribution of GALA+’s core components, and Table 4 further ablates STRIX’s three structural scoring dimensions.

STRIX

GALA+ w/o STRIX

AC@1 100 80 60 40 20

GALA+ w/o Phase III

MRR

100

0.9

90

0.8

80

0.7

70 60

OB

GALA+

AC@3

MRR

O

Score (%)

TT

E

Score (%)

OB

Method

TT

50

0.6 0.5

OB

TT

0.4

OB

TT

Figure 4: Ablation study of GALA+. Table 4: Ablation study of STRIX components. Dataset

Method

AC@1 (%)

AC@3 (%)

MRR

OB

STRIX w/o Instability STRIX w/o Centrality STRIX w/o Impact STRIX

18.18 44.32 36.36 51.14

78.41 75.00 69.32 87.50

0.449 0.628 0.559 0.697

TT

STRIX w/o Instability STRIX w/o Centrality STRIX w/o Impact STRIX

9.09 26.14 19.32 50.00

39.77 68.18 69.32 63.64

0.307 0.498 0.487 0.613

GALA+ Component Ablation. Compared to full GALA+, using STRIX alone reduces AC@1 to 51.14% on OB and 50.00% on TT, showing that trace-based structural scoring is informative but not sufficient on its own. Removing STRIX causes a substantial drop on OB (from 74.44% to 35.56%) and a smaller drop on TT (from 73.33% to 68.89%), indicating that its contribution is dataset-dependent. In particular, STRIX is especially valuable on OB, where metric-only signals are more easily dominated by noisy downstream effects and thus benefit more from trace- and structure-aware ranking. Removing Phase III (Section 3.3) also degrades performance, showing that graph-guided agentic investigation consistently refines the initial ranking by re-examining top candidates through localized multi-modal evidence. Agent-Level Ablation. The ablations above remove entire phases. To test whether GALA+’s LLM agents add value beyond deterministic aggregation over the same inputs, we replace each agent with a fixed rule on OB, leaving the rest of the pipeline unchanged. Substituting reciprocal-rank fusion for the Consolidation Agent lowers AC@1 from 74.44% to 63.33%, AC@3 from 98.89% to 96.70%, and MRR from 0.854 to 0.790. When the Evidence Synthesis Agent is replaced with threshold-count aggregation over DeepDive results in full GALA+, performance similarly drops to 0.790 MRR and 63.33% AC@1. These results indicate that both the Consolidation Agent and Evidence Synthesis Agent contribute beyond simple aggregation rules. STRIX Dimension Ablation. Among the three dimensions, Instability (𝜙 1 ) is the most discriminative: removing it drops AC@1 from 51.14% to 18.18% on OB and from 50.00% to 9.09% on TT. Removing Centrality (𝜙 2 ) or Impact (𝜙 3 ) causes smaller but still clear degradations, confirming that both provide complementary structural and performance signals beyond tail-latency instability alone. On TT, STRIX without Centrality or Impact still attains relatively high AC@3 (68.18% and 69.32%), suggesting that Instability is already effective at retaining the true root cause within the top candidates, while 𝜙 2 and 𝜙 3 are especially important for sharpening the final top-1 ranking.

ASE ’26, October 12–16, 2026, Munich, Germany

Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and Hans-Arno Jacobsen

Table 5: Impact of LLM choice on GALA+ (OB dataset). AC in %, Token = total tokens, $ = cost per incident. Best in bold. LLM

AC@1 AC@3 MRR Token Time (s)

gemini-2.5-flash deepseek-v3 claude-sonnet-4-5 GPT-4.1-mini Qwen3-80B llama-4-maverick gpt-4.1

55.6 67.8 71.1 74.4 76.7 77.8 78.9

87.8 94.4 93.3 98.9 95.6 97.8 94.4

.718 17.8K .810 35.2K .827 45.6K .854 32.7K .855 40.2K .872 32.0K .873 34.0K

$

18.5 64.9 101.3 52.9 21.3 37.4 48.4

.030 .044 .255 .022 .015 .003 .110

Each major component of GALA+ contributes meaningfully to RCA performance: STRIX supplies complementary trace- and structure-aware ranking signals, and graph-guided agentic investigation further sharpens the already strong initial hypotheses. 5.3.4 RQ4: Impact of LLM Selection. Table 5 reports accuracy, total token consumption, latency, and cost per incident for seven LLMs on OB. AC@1 broadly increases with model capability, from gemini2.5-flash (55.6%) to gpt-4.1 (78.9%). Two open-source models—llama4-maverick (77.8%) and Qwen3-80B (76.7%)—approach top-tier accuracy at a fraction of the cost ($0.003 and $0.015 per incident via Together AI), with llama-4-maverick offering the best overall trade-off: near-top accuracy at the lowest cost among all evaluated models. We designate GPT-4.1-mini as the default backbone for reproducibility via its official API, noting that it still achieves 74.4% AC@1 at $0.022— only 4.5 points below gpt-4.1 at one-fifth the cost. Wall-clock time varies considerably (18.5s–101.3s), driven primarily by API response latency rather than token volume; practitioners with strict latency constraints may prefer Qwen3-80B (21.3s) or gemini-2.5-flash (18.5s). GALA+ benefits from stronger LLMs, but open-source models can deliver near-top performance at substantially lower cost.

AC@1 (%)

100

Time (s)

(b) Pruning θ

(c) Batch Size

120

90

100

80

80

70

60

60

40

50 2

5

8

Parallel Branches

0.3

0.6

0.9

Threshold θ

1

2

3

GALA+ is robust across a broad range of parameter settings, and the default configuration provides a strong balance between candidate coverage and inference efficiency.

6

Case Study Root Cause

order-service • cpu (Silent saturation, no error logs)

Initial Hypothesis

Metric-based ranking with BARO: 1. config-service • latency 2. route-plan • memory 3. cancel-service • cpu ... 5. order-service • cpu

Trace-based ranking with STRIX: 1. auth-service 2. user-service 3. station-service ... 7. order-service

Consolidation Agent produces fused ranking RC: 1. stationservice; 2. order-service; 3. auth-service; 4. config-service; ...

Time (s)

AC@1 (%)

(a) Branches

Pruning threshold (𝜃 ). Performance peaks around 𝜃 =0.6. Lower confidence thresholds cause agents to terminate too early, before following potentially informative causal chains, whereas higher thresholds encourage unnecessary graph expansion and slightly reduce accuracy while increasing latency. Batch size (𝑏). A small batch size works best, with performance peaking around 𝑏=2. Smaller batches may limit local coverage, whereas larger batches introduce noisier neighbors and reduce the focus of investigation. Execution time varies only modestly across settings. Neighbor selection. GALA+ samples a batch of 𝑏 neighbors uniformly at random during agentic graph-guided investigation. We compare this against an evidence-driven alternative that orders the neighbors of a service by their STRIX ranking and expands the top 𝑏. On a 30-case OB subset with five seeds, random sampling attains AC@1 0.76 ± 0.03, AC@3 0.97 ± 0.02, and MRR 0.87 ± 0.02, against 0.72 ± 0.04, 0.98 ± 0.02, and 0.85 ± 0.02 for STRIX-ordered expansion. The STRIX-based expansion did not dominate random sampling because STRIX is already used during initial hypothesis generation. Reusing STRIX again for graph expansion can reinforce the same trace-based bias, whereas random sampling provides more diverse local exploration around the top candidates.

4

Batch Size b

Figure 5: GALA+ parameter study results. 5.3.5 RQ5: Impact of Agentic Workflow Parameters. Figure 5 reports AC@1 and execution time of GALA+ on TT with different combinations of the three key agentic parameters: parallel branches 𝑘, pruning threshold 𝜃 , and batch size 𝑏. Parallel branches (𝑘). A moderate value of 𝑘 yields the best accuracy, whereas both smaller and larger settings perform worse. This suggests that investigating too few initial candidates limits coverage, while too many introduce noisier hypotheses without improving the final ranking. Execution time remains largely unchanged across settings because these investigation branches are launched in parallel.

Graph-Guided Agentic Investigation Top k = 5 candidates investigated concurrently (four shown below)

order-service

ĉ = 0.9

3–4× CPU spike, no error logs → silent process overload.

auth-service

ĉ = 0.75

Moderate CPU (~15%), no spikes → Normal processing load.

station-service

ĉ = 0.85

Stable memory usage, high score from centrality alone → no causal relationship.

config-service

ĉ = 0.3

Metric ranked 1st but latency flat at 9.27 ms, no anomaly in logs. Neighbor expansion

Diagnostic Synthesis: order-service CPU saturation is the sole root cause. The false positives are correctly suppressed. True root cause identification is improved from: Final 5th (metric-based) → 2nd (fused ranking) → 1st (Final ranking). Output Incident summarization and action recommendation achieve high SURE-Score and other scores.

Figure 6: GALA+’s reasoning chain for a CPU hog fault case in order-service, ranked top-1 with optimal SURE-Score.

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

We illustrate GALA+ on a CPU-hog fault occurring in order-service of TT (Figure 6). The coarse metric-based ranking places order-service at 5th, while STRIX ranks it 7th based on trace- and structure-aware signals. Although neither modality alone places the true root cause at the top, the fused initial ranking 𝑅 C already promotes order-service to 2nd, indicating that Phase I identifies a high-quality set of plausible hypotheses. Phase III then resolves the remaining ambiguity through graph-guided investigation. The order-service branch finds the clearest evidence of the true fault, namely a 3–4× CPU spike with no error logs, consistent with silent saturation. By contrast, other high-ranked candidates are ruled out through localized multi-modal inspection: their signals are either stable, weak, or unsupported by a causal explanation. After graph-guided investigation and evidence synthesis, GALA+ correctly promotes order-service to rank 1 and suppresses the false positives. This case highlights how graph-guided investigation refines an already strong initial ranking into the correct final diagnosis.

7

Related Work

LLM for RCA. Language-model-based RCA falls into three paradigms. Fine-tuning approaches [1, 8, 36, 37, 46, 64] adapt pre-trained models to RCA-specific incident profiles. Embeddingbased methods [7, 20, 23, 46, 77] use vector representations from models like BERT [13] to capture semantic information in telemetry. Prompt-based methods apply in-context learning [45, 74], chainof-thought reasoning [1, 8, 39, 60, 65, 70], and retrieval-augmented generation [22, 25, 27, 28, 58] to guide reasoning over heterogeneous evidence. Among agentic approaches, Xu et al. [65] introduce RCAAgent together with the OpenRCA benchmark, showing both the promise of LLM-based RCA and the substantial challenges that remain even for multi-agent, execution-based reasoning. RCAgent [59] achieves strong results through tool-augmented reasoning over logs and code, naturally complementing trace- and metric-driven fault localization rather than targeting it directly. Flow-of-Action [39] encodes standard operating procedures into structured workflows that improve reproducibility on anticipated fault patterns, while naturally emphasizing consistency over flexible exploration in less expected scenarios. mABC [73] introduces blockchain-inspired multi-agent collaboration for accurate ranking, though our experiments suggest that its decentralized coordination can occasionally lead to less grounded predictions. GALA [55] orchestrates a ReAct-style investigation reasoning loop after formulating an initial hypothesis. In contrast, GALA+ performs graph-guided multi-modal investigation and generates causally grounded incident responses. Non-LLM RCA Methods. These methods typically rely on causal inference and graph-based analysis. Granger causality [30] and structural equation modeling [50, 52, 63] identify relationships in metric time series, while graph-based methods [41, 61] exploit service topology through centrality and ranking algorithms [5, 24]. These approaches are often limited to a single modality, make limited use of log semantics, and provide little explanatory support for incident response. More recent multi-modal methods address some of these limitations by fusing heterogeneous observability signals [32, 53, 54]. In particular, Nezha [69] represents all three modalities as event graphs, MSoFSAnomaly [17] selects faultsensitive features to reduce redundancy, and MULAN [75] applies

ASE ’26, October 12–16, 2026, Munich, Germany

contrastive learning for multi-modal fusion. GALA+ preserves each modality in a structured diagnostic bundle and couples statistical hypothesis generation with graph-guided LLM reasoning.

8

Threats to Validity

Internal Threats. LLM non-determinism at temperature 1.0 introduces output variability; we mitigate this with standardized prompts and report results averaged over three runs. SURE-Score’s checklist was co-developed with experts from a single organization (Company A), potentially limiting generalizability, though our dual-evaluator and human validation results suggest the dimensions transfer across grading contexts. GALA+ requires access to all three telemetry modalities to achieve expected performance. External Threats. Our evaluation uses controlled fault-injection benchmarks with predefined fault types; real-world incidents may exhibit more complex or concurrent failure modes. Prompt templates are designed for the evaluated datasets’ telemetry schema; adapting to different deployments requires additional prompt engineering. Token cost scales with the number of candidate services and exploration coverage, which may constrain applicability to very large service graphs.

9

Conclusions

This paper presents GALA+, a graph-augmented LLM agentic framework for microservice root cause analysis and incident response generation. By combining dual-signal hypothesis generation via metrics-based ranking and STRIX with graph-guided investigation over localized multi-modal evidence, GALA+ achieves the best overall RCA performance on two benchmarks and outperforms the strongest LLM-based baseline by over 25% in AC@1; this advantage is statistically significant under McNemar’s test (𝑝 < 0.001). Our SURE-Score framework, supported by evaluator LLMs and SRE experts, further confirms substantial gains in diagnostic precision and operational feasibility, while identifying evidence groundedness as an open challenge for current LLM-based RCA methods. These results suggest that topology-constrained reasoning is a promising alternative to unconstrained agentic exploration in domains with rich relational structure. Future work will extend GALA+ to concurrent multi-fault scenarios and incorporate improved context engineering for stronger evidence grounding.

Data Availability The software underlying this work was developed under a collaborative research agreement with our industrial partner, whose terms preclude public release of the source code. To support reproducibility, the paper reports the complete algorithm specifications, system parameters, workload configurations, and hardware setup used in all experiments. Aggregate measurement data are available from the corresponding author on reasonable request, subject to the partner’s approval. The technical report and artifacts permitted for release are available at: https://github.com/EvanTian233/GALA-for-RCA.

Acknowledgments This research was undertaken, in part, thanks to funding from Bell Canada, NSERC, the Canada Research Chairs Program, and ORF.

ASE ’26, October 12–16, 2026, Munich, Germany

References [1] Toufique Ahmed, Supriyo Ghosh, Chetan Bansal, Thomas Zimmermann, Xuchao Zhang, and Saravan Rajmohan. 2023. Recommending root-cause and mitigation steps for cloud incidents using large language models. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1737–1749. [2] Anthropic. 2025. Claude-sonnet-4-5 Release. https://www.anthropic.com/news/ claude-sonnet-4-5 [3] Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72. [4] P Bhosale. 2022. Metrics, Logs, and Traces: A Unified Approach to Observability in Microservices. Journal of Artificial Intelligence, Machine Learning and Data Science 1 (2022), 2084–2088. [5] Monica Bianchini, Marco Gori, and Franco Scarselli. 2005. Inside pagerank. ACM Transactions on Internet Technology (TOIT) 5, 1 (2005), 92–128. [6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901. [7] Sarthak Chakraborty, Shubham Agarwal, Shaddy Garg, Abhimanyu Sethia, Udit Narayan Pandey, Videh Aggarwal, and Shiv Saini. 2023. ESRO: Experience Assisted Service Reliability against Outages. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). 255–267. [8] Yinfang Chen, Huaibing Xie, Minghua Ma, Yu Kang, Xin Gao, Liu Shi, Yunjie Cao, Xuedong Gao, Hao Fan, Ming Wen, et al. 2024. Automatic root cause analysis via large language models for cloud incidents. In Proceedings of the Nineteenth European Conference on Computer Systems. 674–688. [9] Zhi-Yuan Chen, Hao Wang, Xinyu Zhang, Enrui Hu, and Yankai Lin. 2025. Beyond the surface: Measuring self-preference in llm judgments. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 1653–1672. [10] Seong Yeub Chu, Jong Woo Kim, and Mun Yong Yi. 2025. Think together and work better: Combining humans’ and LLMs’ think-aloud outcomes for effective text evaluation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–23. [11] Gordon V Cormack, Charles LA Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval. 758–759. [12] DeepSeek. 2025. deepseek-ai/DeepSeek-V3.1. https://www.together.ai/models/ deepseek-v3-1 [13] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). 4171–4186. [14] Nicola Dragoni, Saverio Giallorenzo, Alberto Lluch Lafuente, Manuel Mazzara, Fabrizio Montesi, Ruslan Mustafin, and Larisa Safina. 2017. Microservices: yesterday, today, and tomorrow. Present and ulterior software engineering (2017), 195–216. [15] Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130 (2024). [16] Aoyang Fang, Songhan Zhang, Yifan Yang, Haotong Wu, Junjielong Xu, Xuyang Wang, Rui Wang, Manyi Wang, Qisheng Lu, and Pinjia He. 2026. Rethinking the Evaluation of Microservice RCA with a Fault Propagation-Aware Benchmark. Proceedings of the ACM on Software Engineering FSE (2026). [17] Nan Fu, Guang Cheng, Guangye Dai, Hantao Mei, Xing Qiu, and Yue Teng. 2025. A failure analysis framework to provide pure anomalous data using multi-source data of fault-sensitive microservices. Journal of Systems and Software 230 (2025), 112513. doi:10.1016/j.jss.2025.112513 [18] Google. 2025. Gemini-2.5-flash Model Card. https://docs.cloud.google.com/vertexai/generative-ai/docs/models/gemini/2-5-flash [19] Google. 2025. Gemini-2.5-pro Model Card. https://docs.cloud.google.com/vertexai/generative-ai/docs/models/gemini/2-5-pro [20] Guijiao He. 2023. Construction and Experimental Evaluation of Document Causality Extraction Model Based on CGCN-BERT. In 2023 International Conference on Applied Intelligence and Sustainable Computing (ICAISC). 1–6. [21] Azam Ikram, Sarthak Chakraborty, Subrata Mitra, Shiv Saini, Saurabh Bagchi, and Murat Kocaoglu. 2022. Root cause analysis of failures in microservices through causal discovery. Advances in Neural Information Processing Systems (2022). [22] Yuxuan Jiang, Chaoyun Zhang, Shilin He, Zhihao Yang, Minghua Ma, Si Qin, Yu Kang, Yingnong Dang, Saravan Rajmohan, Qingwei Lin, et al. 2024. Xpert: Empowering incident management with query recommendations via large language models. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13.

Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and Hans-Arno Jacobsen

[23] Jinxi Kuang, Jinyang Liu, Junjie Huang, Renyi Zhong, Jiazhen Gu, Lan Yu, Rui Tan, Zengyin Yang, and Michael R Lyu. 2024. Knowledge-aware alert aggregation in large-scale cloud systems: a hybrid approach. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice. 369–380. [24] Gregory F Lawler and Vlada Limic. 2010. Random walk: a modern introduction. Vol. 123. Cambridge University Press. [25] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33 (2020), 9459–9474. [26] Mingjie Li, Zeyan Li, Kanglin Yin, Xiaohui Nie, Wenchi Zhang, Kaixin Sui, and Dan Pei. 2022. Causal inference-based root cause analysis for online service systems with intervention recognition. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 3230–3240. [27] Peiwen Li, Xin Wang, Zeyang Zhang, Yuan Meng, Fang Shen, Yue Li, Jialong Wang, Yang Li, and Wenwu Zhu. 2024. RealTCD: temporal causal discovery from interventional data with large language model. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. 4669–4677. [28] Yichen Li, Yulun Wu, Jinyang Liu, Zhihan Jiang, Zhuangbin Chen, Guangba Yu, and Michael R. Lyu. 2025. COCA: Generative Root Cause Analysis for Distributed Systems with Code Knowledge. In Proceedings of the IEEE/ACM 47th International Conference on Software Engineering. 1346–1358. [29] Zeyan Li, Junjie Chen, Rui Jiao, Nengwen Zhao, Zhijun Wang, Shuwei Zhang, Yanjun Wu, Long Jiang, Leiqin Yan, Zikai Wang, et al. 2021. Practical root cause localization for microservice systems via trace analysis. In 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS). 1–10. [30] Cheng-Ming Lin, Ching Chang, Wei-Yao Wang, Kuang-Da Wang, and Wen-Chih Peng. 2024. Root Cause Analysis in Microservice Using Neural Granger Causal Discovery. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 206–213. [31] Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out (Workshop at ACL 2004). 74–81. [32] Fengrui Liu, Yang Wang, Zhenyu Li, Rui Ren, Hongtao Guan, Xian Yu, Xiaofan Chen, and Gaogang Xie. 2022. Microcbr: Case-based reasoning on spatio-temporal fault knowledge graph for microservices troubleshooting. In International Conference on Case-Based Reasoning. 224–239. [33] Meta. 2025. meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8. https://www.together.ai/models/llama-4-maverick [34] OpenAI. 2025. GPT-4.1-mini Model Card. https://developers.openai.com/api/ docs/models/gpt-4.1-mini [35] OpenAI. 2025. GPT-4.1 Model Card. https://platform.openai.com/docs/models/gpt4.1 [36] Yue Pang, Min Zhang, Yanli Liu, Xiangbin Li, Yidi Wang, Yahang Huan, Zhuo Liu, Jin Li, and Danshi Wang. 2024. Large language model-based optical network log analysis using LLaMA2 with instruction tuning. Journal of Optical Communications and Networking 16, 11 (2024), 1116–1132. [37] Zhendong Pang, Hao Zhang, and Teng Li. 2024. Hybrid Fine-Tuning in Large Language Model Learning for Machinery Fault Diagnosis. In 2024 IEEE 22nd International Conference on Industrial Informatics (INDIN). 1–6. [38] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL). 311–318. [39] Changhua Pei, Zexin Wang, Fengrui Liu, Zeyan Li, Yang Liu, Xiao He, Rong Kang, Tieying Zhang, Jianjun Chen, Jianhui Li, Gaogang Xie, and Dan Pei. 2025. Flow-of-Action: SOP Enhanced LLM-Based Multi-Agent System for Root Cause Analysis. In Companion Proceedings of the ACM on Web Conference 2025. 422–431. doi:10.1145/3701716.3715225 [40] Matilda QR Pembury Smith and Graeme D Ruxton. 2020. Effective use of the McNemar test. Behavioral Ecology and Sociobiology 74, 11 (2020), 133. [41] Luan Pham, Huong Ha, and Hongyu Zhang. 2024. Baro: Robust root cause analysis for microservices via multivariate bayesian online change point detection. Proceedings of the ACM on Software Engineering 1, FSE (2024), 2214–2237. [42] Luan Pham, Huong Ha, and Hongyu Zhang. 2024. Root Cause Analysis for Microservice System based on Causal Inference: How Far Are We?. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 706–715. [43] Luan Pham, Hongyu Zhang, Huong Ha, Flora Salim, and Xiuzhen Zhang. 2025. RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Data. In The 2025 ACM Web Conference (WWW). 777–780. [44] Qwen. 2025. Qwen/Qwen3-Next-80B-A3B-Instruct. https://www.together.ai/ models/qwen3-next-80b-a3b-instruct [45] Devjeet Roy, Xuchao Zhang, Rashi Bhave, Chetan Bansal, Pedro Las-Casas, Rodrigo Fonseca, and Saravan Rajmohan. 2024. Exploring llm-based agents for root cause analysis. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering. 208–219. [46] Amrita Saha and Steven CH Hoi. 2022. Mining root cause knowledge from cloud service incident investigations for aiops. In Proceedings of the 44th international conference on software engineering: Software engineering in practice. 197–206.

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

[47] Gerard Salton and Michael J. McGill. 1986. Introduction to modern information retrieval. McGraw-Hill, Inc. [48] Friedrich Schmid and Rafael Schmidt. 2007. Multivariate extensions of Spearman’s rho and related statistics. Statistics & probability letters 77, 4 (2007), 407–416. [49] Pranab Kumar Sen. 1968. Estimates of the regression coefficient based on Kendall’s tau. Journal of the American statistical association 63, 324 (1968), 1379–1389. [50] Shohei Shimizu, Patrik O. Hoyer, Aapo Hyvärinen, and Antti Kerminen. 2006. A Linear Non-Gaussian Acyclic Model for Causal Discovery. Journal of Machine Learning Research 7 (2006), 2003–2030. [51] Benjamin H. Sigelman, Luiz André Barroso, Mike Burrows, Pat Stephenson, Manoj Plakal, Donald Beaver, Saul Jaspan, and Chandan Shanbhag. 2010. Dapper, a Large-Scale Distributed Systems Tracing Infrastructure. Technical Report. Google, Inc. http://research.google.com/archive/papers/dapper-2010-1.pdf [52] Peter Spirtes, Clark Glymour, and Richard Scheines. 2001. Causation, prediction, and search. MIT press. [53] Yongqian Sun, Zihan Lin, Binpeng Shi, Shenglin Zhang, Shiyu Ma, Pengxiang Jin, Zhenyu Zhong, Lemeng Pan, Yicheng Guo, and Dan Pei. 2025. Interpretable failure localization for microservice systems based on graph autoencoder. ACM Transactions on Software Engineering and Methodology 34, 2 (2025), 1–28. [54] Lei Tao, Shenglin Zhang, Zedong Jia, Jinrui Sun, Minghua Ma, Zhengdan Li, Yongqian Sun, Canqun Yang, Yuzhi Zhang, and Dan Pei. 2024. Giving Every Modality a Voice in Microservice Failure Diagnosis via Multimodal Adaptive Optimization. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 1107–1119. [55] Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, and Hans-Arno Jacobsen. 2025. GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis? arXiv preprint arXiv:2508.12472 (2025). [56] Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and HansArno Jacobsen. 2026. GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices [Technical Report]. https: //github.com/EvanTian233/GALA-for-RCA/blob/main/techinical_report.pdf. [57] Together AI. 2026. Together AI Pricing. https://www.together.ai/pricing [58] Zexin Wang, Jianhui Li, Minghua Ma, Ze Li, Yu Kang, Chaoyun Zhang, Chetan Bansal, Murali Chintalapati, Saravan Rajmohan, Qingwei Lin, et al. 2024. Large Language Models Can Provide Accurate and Interpretable Incident Triage. In 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE). 523–534. [59] Zefan Wang, Zichuan Liu, Yingying Zhang, Aoxiao Zhong, Jihong Wang, Fengbin Yin, Lunting Fan, Lingfei Wu, and Qingsong Wen. 2024. Rcagent: Cloud root cause analysis by autonomous agents with tool-augmented large language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. 4966–4974. [60] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837. [61] Li Wu, Johan Tordsson, Erik Elmroth, and Odej Kao. 2020. Microrca: Root cause localization of performance issues in microservices. In NOMS 2020-2020 IEEE/IFIP Network Operations and Management Symposium. 1–9. [62] Zhiqiang Xie, Yujia Zheng, Lizi Ottens, Kun Zhang, Christos Kozyrakis, and Jonathan Mace. 2024. Cloud Atlas: Efficient Fault Localization for Cloud Systems using Language Models and Causal Insight. arXiv preprint arXiv:2407.08694 (2024). [63] Ruyue Xin, Peng Chen, and Zhiming Zhao. 2023. Causalrca: Causal inference based precise fine-grained root cause localization for microservice applications. Journal of Systems and Software 203 (2023), 111724. [64] Xueyang Xing, Bo Jia, Zhicheng Huang, Yongzhi Chen, Junjie Wang, Anfei Fan, Xin Chen, and Lei Cao. 2023. A fusion inference method for large language models and knowledge graphs based on structured injection and causal inference.

ASE ’26, October 12–16, 2026, Munich, Germany

In Proceedings of the 2023 5th International Conference on Internet of Things, Automation and Artificial Intelligence. 208–213. [65] Junjielong Xu, Qinan Zhang, Zhiqing Zhong, Shilin He, Chaoyun Zhang, Qingwei Lin, Dan Pei, Pinjia He, Dongmei Zhang, and Qi Zhang. 2025. OpenRCA: Can Large Language Models Locate the Root Cause of Software Failures?. In The Thirteenth International Conference on Learning Representations. [66] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems 36 (2023), 11809–11822. [67] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In The Eleventh International Conference on Learning Representations. [68] Guangba Yu, Pengfei Chen, Pairui Li, Tianjun Weng, Haibing Zheng, Yuetang Deng, and Zibin Zheng. 2023. Logreducer: Identify and reduce log hotspots in kernel on the fly. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 1763–1775. [69] Guangba Yu, Pengfei Chen, Yufeng Li, Hongyang Chen, Xiaoyun Li, and Zibin Zheng. 2023. Nezha: Interpretable fine-grained root causes analysis for microservices on multi-modal observability data. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 553–565. [70] Dylan Zhang, Xuchao Zhang, Chetan Bansal, Pedro Las-Casas, Rodrigo Fonseca, and Saravan Rajmohan. 2024. LM-PACE: Confidence estimation by large language models for effective root causing of cloud incidents. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering. 388–398. [71] Shenglin Zhang, Sibo Xia, Wenzhao Fan, Binpeng Shi, Xiao Xiong, Zhenyu Zhong, Minghua Ma, Yongqian Sun, and Dan Pei. 2025. Failure diagnosis in microservice systems: A comprehensive survey and analysis. ACM Transactions on Software Engineering and Methodology 35, 1 (2025), 1–55. [72] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations (ICLR). [73] Wei Zhang, Hongcheng Guo, Jian Yang, Zhoujin Tian, Yi Zhang, Yan Chaoran, Zhoujun Li, Tongliang Li, Xu Shi, Liangfan Zheng, and Bo Zhang. 2024. mABC: Multi-Agent Blockchain-inspired Collaboration for Root Cause Analysis in Micro-Services Architecture. In Findings of the Association for Computational Linguistics: EMNLP 2024. 4017–4033. [74] Xuchao Zhang, Supriyo Ghosh, Chetan Bansal, Rujia Wang, Minghua Ma, Yu Kang, and Saravan Rajmohan. 2024. Automated root causing of cloud incidents using in-context learning with GPT-4. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering. 266–277. [75] Lecheng Zheng, Zhengzhang Chen, Jingrui He, and Haifeng Chen. 2024. MULAN: Multi-modal Causal Structure Learning and Root Cause Analysis for Microservice Systems. In Proceedings of the ACM Web Conference 2024. 4107–4116. doi:10.1145/3589334.3645442 [76] Zhouruixing Zhu, Cheryl Lee, Xiaoying Tang, and Pinjia He. 2024. HeMiRCA: Fine-grained root cause analysis for microservices with heterogeneous data sources. ACM Transactions on Software Engineering and Methodology (2024). [77] Qingheng Zhuang, Li Zhu, and Sen Lin. 2022. A GAN-Bert Based Fault Diagnosis Model for CBTC Data Communication Systems Using Edge-to-edge Collaboration Training. In ICC 2022-IEEE International Conference on Communications. 5076–5081.

Received 2026-03-26; accepted 2026-06-18

Related documents

• Characterizing the Failure Modes of LLMs in Resolving Real-World GitHub Issues#178935
• DeepFaith: Evidence-Grounded LLMs for Faithful Incident Reporting in Multi-Stage APT Defense#405542
• DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction#429298
• Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents#1132312
• On the Cohesion of Hub-like microservices#648537