Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

arXiv:2611.02416v2 [cs.SE] 3 Nov 2026

KYOOHYUN LEE, Hanyang University, Republic of Korea WOOHYEOK PARK, Hanyang University, Republic of Korea TAEYOON KIM, Hanyang University, Republic of Korea SUNGYONG BAIK, Hanyang University, Republic of Korea KYUNGYONG LEE, Hanyang University, Republic of Korea Root cause analysis (RCA) in microservice systems requires distinguishing failure sources from routine fluctuations and propagated symptoms. Small language models (SLMs) offer a practical solution under cost and privacy constraints, and recent approaches improve their capabilities through structured diagnostic procedures. However, performing diagnostic analysis does not ensure correct root-cause identification. Our empirical study shows that final decisions often favor large anomaly scores despite the model’s own propagation analysis suggesting otherwise. Moreover, the input does not reveal whether these apparently anomalous signals are common during normal operation. These findings motivate our system, which improves how SLMs assess evidence and select root causes through Historical Frequency Profiling and Biased-Decision Correction. Historical Frequency Profiling quantifies how often metric and log signals are flagged as anomalous in the fault-free period, providing context for distinguishing routine deviations from potential failure evidence. Biased-Decision Correction identifies points where the model reaches different conclusions from the same initial analysis and uses preference optimization to favor more accurate diagnoses. Against baselines including the 14B OpsAgent and RCA-Agent with commercial models, our system with a 4B model achieves the highest or tied-highest accuracy on all four datasets, reaching 48.7 % on Bank compared with 18.5 % for OpsAgent.

1

Introduction

Modern cloud services consist of interdependent microservices, where a fault in one component can produce symptoms across many others [39, 51, 65]. Root Cause Analysis (RCA) distinguishes the failure source from propagated symptoms and identifies the faulty component, fault type, and occurrence time [52]. This requires interpreting telemetry of metrics, logs, and traces, and its volume and complexity quickly exceed what on-call engineers can manually inspect. Classical RCA methods analyze statistical dependencies and event patterns to localize failure sources [31, 33, 54]. Recent work has shifted attention to large language models (LLMs), which interpret telemetry and explain how observations support a diagnosis [32, 51, 52]. To fit telemetry into a model’s limited context window, RCA systems typically retain signals flagged as anomalous and summarize them as a text prompt, a step we refer to as data processing [32, 52]. This step determines the evidence available to the model, which examines it alongside component dependencies to identify the root cause. However, API costs and restrictions on sharing operational telemetry motivate self-hosted small language models (SLMs) [5, 32, 60]. Their diagnostic accuracy remains limited. Given prompts built by existing data processing, the off-the-shelf SLMs we evaluate correctly answer all requested elements for fewer than 4 % of incidents. Frontier models achieve nearly three times the accuracy on the same prompts, at substantially higher inference costs (§2.3). Recent approaches improve diagnosis by organizing how models investigate incidents. RCAAgent queries telemetry through tools [52], while SLM-oriented systems such as OpsAgent [32], Authors’ Contact Information: Kyoohyun Lee, khyhlee@hanyang.ac.kr, Department of Artificial Intelligence, Hanyang University, Seoul, Republic of Korea; Woohyeok Park, woohyeok@hanyang.ac.kr, Department of Data Science, Hanyang University, Seoul, Republic of Korea; Taeyoon Kim, tykim7@hanyang.ac.kr, Department of Data Science, Hanyang University, Seoul, Republic of Korea; Sungyong Baik, dsybaik@hanyang.ac.kr, Hanyang University, Seoul, Republic of Korea; Kyungyong Lee, kyungyong@hanyang.ac.kr, Department of Data Science, Hanyang University, Seoul, Republic of Korea.

2

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

FoundRoot [51], and ThinkFL [60] guide or train models to examine observations, analyze propagation, and refine diagnoses. However, a model that follows such a procedure can still select the wrong root cause. It may note that a large anomaly could be routine behavior or a symptom propagated from another component, yet still choose the component showing that anomaly. We investigate this gap through an empirical study of data processing and Qwen3-4B’s diagnostic decisions on the Bank dataset of OpenRCA [52], revealing two limitations. The input lacks normal-behavior context. The examined data processing flags signals with shortterm statistics and fixed thresholds, so fluctuations during normal operation are flagged as well. For example, garbage collection causes a JVM memory metric on Bank to be flagged in 97 % of fault-free windows. Such signals still reach the prompt as failure evidence, with large anomaly scores and no indication of how common they are. As a result, the actual root-cause component has the largest anomaly score in only 8 % of incident windows, even though it is among the candidates in 99 % (§3.1). The final decision often departs from the model’s own analysis. Qwen3-4B examines multiple candidates and propagation relationships in all 640 trajectories, yet it justifies its selection by large anomaly scores in 76 % of them and departs from its own propagation analysis in 68 %. Adding a few-shot example with a correct diagnosis of another incident does not reliably fix this behavior. The gains depend largely on whether the example names the same root-cause component as the target incident (§3.2). These findings reveal a gap between performing diagnostic analysis and selecting the root cause. Building on this insight, we present an SLM-based RCA system that addresses this gap by providing normal-behavior context for assessing anomalies and targeting decisions where the model reaches different conclusions from the same initial analysis. Historical Frequency Profiling measures how often metric and log signals are flagged as anomalous in the fault-free period. This frequency, called the Expected Outlier Baseline (EOB), accompanies anomaly scores and guides candidate ordering. Traces are summarized by their latency relative to that in the fault-free period. These profiles help distinguish routine deviations from potential failure evidence without treating rarity alone as proof of a root cause. Biased-Decision Correction compares the model’s own responses that share the same initial analysis but reach different diagnostic conclusions. Preference pairs reflect correctness on individual RCA elements and favor more complete diagnoses among partially correct responses. When none of the responses is correct, auxiliary hints help obtain a correct one, which is checked against the original evidence. The hints are removed before training. Direct Preference Optimization (DPO) [38] then favors more accurate responses by training on the reasoning and decisions following the shared initial analysis. We evaluate our system on eight datasets. Telecom, Bank, and Market from OpenRCA [52] and AIOps 2025 [35] are used for evaluation within each system, and four held-out datasets are used to test whether an adapter trained on one system transfers to unseen systems. With a 4B model and one inference call per incident, our system achieves the highest exact-match accuracy on Telecom, Bank, and AIOps 2025 and ties with the best baseline on Market. On Bank, it reaches 48.7 %, compared with 18.5 % for OpsAgent-14B [32]. Under identical input, Biased-Decision Correction also enables the 4B model to outperform an off-the-shelf 14B model. A Bank-trained adapter exceeds FoundRoot-14B in exact-match accuracy on the three held-out datasets with full diagnostic labels, using target-system fault-free profiles without further fine-tuning. Table 5 (§6) compares our system with existing approaches in input evidence, diagnostic scope, and training target. In summary, this paper makes the following contributions.

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

Entry point Browsing Checkout & fulfillment Storage Recommendation Product

Catalog

Frontend

3

The cloud service system, cloudbed-1, experienced one failure within the time range of March 20, 2022, from 09:00 to 09:30. […] You are tasked Query with identifying the root cause component and the root cause reason. timestamp,cmdb_id,kpi_name,value

1647738600,node-6.shippingservice-1,container_spec_cpu_shares,204.0

Ads

Metric 1647738600,node-6.shippingservice-1,container_spec_memory_swap_limit_… Shipping

Currency

Cart

Database

Checkout Payment

Cart

Log

log_id,timestamp,cmdb_id,log_name,value

u8LnpH8BDiVcQfZwXDC2,1647739097,shippingservice-1,log_shippingservice… timestamp,cmdb_id,span_id,trace_id,duration,type,status_code,operatio…

Trace 1647738028123,shippingservice-1,a70d0d74f7d4531e,294214803d3826913ac2…

Email

(a) Call graph of Market

(b) Telemetry of shippingservice-1 in Market during an incident

Fig. 1. Inputs of microservice RCA in OpenRCA [52].

• An empirical study revealing missing normal-behavior context and a gap between the evaluated SLM’s diagnostic analysis and final root-cause selection. • An SLM-based RCA system that contextualizes anomaly evidence with fault-free frequencies and targets diagnostic decisions that diverge after the same initial analysis. • An evaluation across eight datasets demonstrating improved accuracy with a 4B model, including comparisons under identical input and transfer without further fine-tuning. 2 Root Cause Analysis in Microservice Systems A microservice system serves each user request through a chain of services, so a fault in one component propagates to the components that depend on it. In the Market system of OpenRCA [52] (Fig. 1a), for example, a slow response of Shipping is also observed at Checkout and Frontend. RCA must therefore distinguish the root-cause component from the components that only show propagated symptoms [39, 51, 60]. RCA relies on three types of telemetry. Metrics are time series recorded periodically, such as CPU utilization, logs are semi-structured text records emitted by components, and traces record each service call of a request as a span with its start time, duration, and status code [52]. Fig. 1b shows the three types for shippingservice-1, an instance of Shipping, during an incident. Because they are recorded separately, diagnosis must link them by time and component. 2.1

Problem Formulation

A microservice system consists of a set of components C, such as hosts, instances, and services (e.g., node-6 and shippingservice-1 in Fig. 1b), and a call graph 𝐺 among them, which is obtained from deployment information or from traces aggregated over a long period. An on-call engineer sets an incident window 𝑊 = [𝑡 0, 𝑡 1 ] as the period in which the failure is suspected [32]. The telemetry observed in 𝑊 is denoted by T𝑊 = (M𝑊 , L𝑊 , R𝑊 ), the sets of metric time series, log entries, and spans in 𝑊 , respectively. Because R𝑊 records only the calls made during 𝑊 , dependencies between components that were not called in 𝑊 are available only through 𝐺. A fault of type 𝑓 ∈ F that occurs at component 𝑐 ∈ C at time 𝑡 is described by the label 𝑦 = (𝑐, 𝑓 , 𝑡), where F is a predefined set of fault types such as high CPU usage and network packet loss. An incident consists of one or more faults within an incident window, with a query 𝑞 that asks for some of these three elements. RCA predicts the requested elements from 𝑞, T𝑊 , and 𝐺 as ˆ 𝑓ˆ, 𝑡ˆ) = Ψ(𝑞, T𝑊 , 𝐺), 𝑐ˆ ∈ C, 𝑓ˆ ∈ F , 𝑡ˆ ∈ 𝑊 . 𝑦ˆ = (𝑐, (1)

4

2.2

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

LLM-based RCA Approaches

Classical RCA methods detect anomalous signals, relate them over the call graph, and rank candidate components, using either statistical scores [31, 33, 48, 57, 58, 66] or models trained on past incidents [26, 28]. Because these methods rank signals by statistical patterns rather than by what a metric, log, or trace means, metric noise and complex fault propagation can mislead their ranking [51]. They also output a ranked list of candidates rather than a diagnosis that relates the evidence to the root cause [32, 60]. LLM-based RCA systems address these limitations [1, 5, 32, 40, 51, 52, 64]. Drawing on the domain knowledge acquired during pre-training, an LLM interprets signals, infers fault propagation along the call graph, and explains its conclusion in natural language, for example through chainof-thought (CoT) reasoning [50]. Because raw telemetry exceeds the context window [52, 60, 61], these systems condense telemetry into a prompt through data processing and guide the model with a diagnostic procedure, given either in the prompt or through training. 2.2.1 Data Processing: Curating Inputs by Anomaly Statistics. Data processing, which selects anomalous signals statistically and converts them into a prompt [32, 44, 51, 52, 60], determines the scope and format of the model input. It first divides telemetry into signals. A metric signal is a time series identified by a component and a key performance indicator (KPI) such as CPU utilization. A log signal is the set of entries of one template emitted by a component, where a template, extracted by a log parser such as Drain [15], is the fixed text of an entry without variable parts such as identifiers and numeric values [17, 21]. A trace signal is the set of spans of all calls to a component. We denote the set of signals in 𝑊 by S𝑊 . Data processing then assigns each signal 𝑠 ∈ S𝑊 an anomaly score 𝜙 (𝑠) and keeps only the signals whose score exceeds a threshold 𝜏, yielding S̃𝑊 = {𝑠 ∈ S𝑊 : 𝜙 (𝑠) > 𝜏 } [32]. For a metric signal, 𝜙 is the z-score |𝑣 − 𝜇|/𝜎 at the point in 𝑊 where the observed value 𝑣 deviates the most from the mean. 𝜇 and 𝜎 are estimated from a short comparison interval, typically the few minutes around 𝑊 , and 𝜏 is typically 3 (the 3-sigma rule) [32, 51]. For a log signal, 𝜙 is the number of entries of the template that the component emits in 𝑊 , counted only for templates that contain keywords such as fatal, error, and crash [32]. For a trace signal, 𝜙 is the number of spans whose latency exceeds a threshold set for their call type [32]. For logs and traces, 𝜏 is 0, so a signal remains if any entry satisfies the condition. The selected signals are then grouped by component, and components with at least one selected signal form the candidate set E𝑊 . Data processing then writes E𝑊 , the selected signals of each candidate with their 𝜙, and 𝐺 as text with render to build the prompt 𝑝, which also contains the query and output format [32, 51]. LLM-based RCA thus splits Ψ in Eq. (1) into two steps: data processing builds 𝑝 from 𝑞, T𝑊 , and 𝐺, and the model outputs 𝑦ˆ from 𝑝. The model can therefore choose only from E𝑊 and judge each candidate only by its selected signals and their 𝜙: a root cause missing from E𝑊 cannot be found, and a misleading 𝜙 can draw the model to a wrong candidate. 2.2.2 In-Context Learning. In-context learning changes the behavior of a model by adding instructions or examples 𝑑 to the prompt while keeping its weights fixed [8]. The model receives 𝑝 ⊕ 𝑑, where ⊕ denotes the concatenation of two token sequences. Without such guidance, the model reaches its conclusion through a different process in each trial, for example by skipping some components or by accepting the first anomaly it finds as the cause [22, 39, 51]. A pre-trained model also has general domain knowledge but lacks knowledge specific to the target system, such as the diagnostic steps that its operators follow [5, 32]. LLM-based RCA systems therefore provide the model with a diagnostic procedure that specifies which components to examine and in what order, such as scanning metrics before analyzing

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

5

propagation along the call graph. The procedure is either stated as instructions [39, 50] or given as the diagnosis records of similar past incidents [5, 32]. 2.2.3 Fine-Tuning: SFT and RL. Fine-tuning updates the weights of a model with examples or rewards [49]. Supervised fine-tuning (SFT) maximizes the likelihood of a target output for each input, and reinforcement learning (RL) maximizes the expected reward of the outputs that the model generates, with algorithms such as PPO [41], GRPO [42], and DAPO [55]. Direct Preference Optimization (DPO) [38] instead trains the model on pairs of preferred and dispreferred responses to the same input, without a separate reward model. For RCA, prior work performs SFT on trajectories that follow a predefined diagnostic procedure and then improves accuracy with RL [32, 51, 60]. Both methods apply the training signal to every token of an output, so they adjust the diagnostic procedure and the final decision together. 2.3

Frontier Models and SLMs for RCA

Deploying LLM-based RCA requires choosing between frontier models accessed through APIs and SLMs that can be self-hosted. We refer to open-weight models with up to 20B parameters that can be served on a single GPU as SLMs. To compare the two, we use the accuracy measure of OpenRCA [52], the fraction of incidents for which all elements, such as component, fault type, and occurrence time, asked by the query are correct. We call this measure exact-match accuracy and denote it by Exact. We ran both types of models on three OpenRCA datasets (Telecom, Bank, and Market) through Amazon Bedrock [2], giving all models the same prompt built by the data processing of OpsAgent [32]. Claude Opus 5 [3] and GPT-5.6 Sol reached an Exact of 10.5 % and 10.6 % at $42.1 and $28.5 per 1,000 incidents, respectively. Off-the-shelf Gemma 3 4B, Gemma 3 12B, and gpt-oss-20b cost only $0.2–0.4 per 1,000 incidents but reached only 3.7–3.9 %. Frontier models are thus more accurate, but their API costs grow with the number of incidents, and they require sending operational data outside the organization [5, 32]. Self-hosted SLMs avoid both problems, so improving their accuracy is the practical path for RCA in production. This requires locating where accuracy is lost, either in the prompt built by data processing or in the cause selection of the SLM, which we examine in the next section. 3

Empirical Study

The accuracy of LLM-based RCA depends on both the input built by data processing [32, 39, 52, 61] and the cause selection of the model on that input [39, 51, 52]. We examine the two separately through the following questions. • Q1. Does the input 𝑝 built by existing data processing contain information that distinguishes the root-cause component from other candidates? (§3.1) • Q2. What limits the accuracy of the SLM in selecting the root cause? (§3.2) We study both questions on the Bank dataset of OpenRCA [52], collected from a banking microservice system. It contains 10 days of metrics, logs, and traces and 136 incidents from injected faults, each with a query that specifies a 30-minute incident window. Bank labels the fault type of every fault and has relatively few components, which suits qualitative analysis. 3.1

Prompts Built by Existing Data Processing

We analyze the prompt built by the data processing of OpsAgent [32], which contains the anomaly scores 𝜙 (𝑠) (z-scores) of metrics, selected log entries, and summaries of high-latency spans. For metrics, 𝜇 and 𝜎 are estimated from a short comparison interval before the incident window 𝑊 , as in existing data processing [32, 51], and a signal is flagged as anomalous when 𝜙 (𝑠) exceeds 𝜏 = 3.

6

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

Cache

memory

JVM

memory

To check whether 𝜙 distinguishes the root cause, we measure how often each signal is flagged as anomalous when no fault is present. We call the time outside all incident windows the fault-free period [54]. Excluding 5 minutes before and after each incident window as a margin, we split this period into 10-minute fault-free windows and apply the same data processing to each of them. Fig. 2 shows two metrics of Tomcat01. JVM μ ± 3σ (5 min) 3σ (10 min) Incident (30 min) Non-incident memory rises and falls every 5 minutes with 10 days of data No3data 3-hour zoom h GB garbage collection (GC) and exceeds 𝜏 in 97 % of 4 Tomcat01 2.4 the fault-free windows. In contrast, cache mem2.0 2 1.6 ory exceeds 𝜏 in only 17 windows over 10 days, 0 Flagged and 13 of them fall within incident windows. GB When cache memory exceeds 𝜏 in 𝑊 , a fault is 4 4.2 3.9 2 therefore the likely cause, yet the prompt presents 3.6 3.3 0 both signals in the same format without showing Flagged how often each exceeds 𝜏 in the fault-free period. Incidents JVM memory is misjudged because the 5 min- March 4–7 9–10 12 23–25 22:30 23:30 00:30 01:30 utes before 𝑊 miss the full range of its GC cycle, which underestimates 𝜎. With 𝜇 and 𝜎 from the Fig. 2. Signals exceeding 𝜏 in fault-free windows vs. only during failures. whole fault-free period instead of the preceding 5 minutes of the incident window, the false positive rate drops from 97 % to 4 %. Logs and traces show the same pattern. For logs, data processing flags entries that contain keywords such as error and failure, and the JVM writes a GC entry such as “[GC (Allocation Failure)...]” for every routine young-generation collection. The GC logs of Tomcat01–04 are therefore flagged as anomalous in 82–87 % of the fault-free windows, but the Tomcat that writes them is the root cause in only 8–15 % of the incident windows where they appear. For traces, data processing counts the high-latency spans of the calls each component receives, such as “Callee=IG01, Count=110, MaxLatency=10865ms”. Because its released implementation sets the latency threshold from the spans in the incident window itself, some spans exceed it in every window. The spans of IG01 and IG02 are flagged as high latency in 97–98 % of the fault-free windows, but IG01 or IG02 is the root cause in only 7–14 % of the incident windows. Such signals act as false positives [54, 62], but they make up most of the log and trace evidence in the prompts. As a result, although the root-cause component is in E𝑊 for 99 % of incident windows, it has the largest 𝜙 for only 8 %. Finding 1. Existing data processing compares each signal against the incident window or the few minutes around it, so the prompt does not show how each signal behaves without faults. In Bank, the root cause is a candidate in 99 % of incident windows but has the largest 𝜙 in only 8 %. Exceeding 𝜏 in 𝑊 alone does not point to the root cause, and the prompt should also state how often each signal exceeds 𝜏 without faults.

3.2

Root Cause Selection by the SLM

Prior work provides a diagnostic procedure in the prompt or trains the model to follow it [32, 51, 60]. We examine whether the SLM performs the diagnostic procedure by itself and whether providing the procedure reduces errors in cause selection. For this analysis, we collected 640 reasoning trajectories that Qwen3-4B [45] generated, eight per incident, from the prompt described above for the 80 Bank incidents in our training split. We then analyzed how candidate examination and intermediate reasoning lead to the final answer. We labeled each trajectory with two independent judgments by Claude Opus 5 as an LLM judge [63] and resolved disagreements with a third. The per-item agreement between the two judgments was 88–100 %, and Cohen’s 𝜅 was 0.68–0.83. The authors then reviewed all final judgments.

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

Case openrca_bank_1616667180_107, trajectory 7.

GT: IG02.

7

Answer: Tomcat01.

Scan Looking at the metrics, Tomcat01, Tomcat02, Tomcat03, and Tomcat04 have several issues. For example, Tomcat01 has high JVM memory usage and disk I/O spikes. [. . . ] IG01 and IG02 have local disk I/O problems, like high DSKBps and DSKWrite.[. . . ] Propagation Looking at the component graph, Tomcat instances depend on MG (MG01, MG02), Redis01, Redis02, and the OS resources. The propagation shows that Tomcat01, 02, 03, 04 are downstream of IG01, IG02, which are downstream of apache01 and 02. So the problem might be starting from the apache servers, which are upstream of the Tomcats. Ranking So the top components would be Tomcat instances, as they have the most direct issues with memory and CPU. Then the disk metrics for IG01 and IG02, which are upstream of Tomcat. Finally, the MG01 and MG02, which are dependencies of Tomcat [. . . ] Answer: 1. Tomcat01: High JVM memory usage and frequent GC errors indicate memory pressure leading to crashes or slowdowns. 2. IG01. 3. MG01.

(a) A final decision that contradicts the intermediate conclusion Case openrca_bank_1614866400_50, trajectory 5.

GT: MG01 (JVM OOM heap).

Answer: Tomcat01.

Input Tomcat01: CurrentThreadsBusy (𝑧=30), sdb_DSKBps (𝑧=30), CPU-2 idle (𝑧=30) [. . . ] IG01: JVM-Memory_7778_HeapMemoryUsed (𝑧=6.0, max 1017709120), JVM-Memory_7778_HeapMemoryUsage (𝑧=5.5), sda_DSKWTps (𝑧=6.1) [. . . ] Reasoning Looking at the component graph, Tomcat01, Tomcat02, Tomcat03, Tomcat04 are dependent on IG01 and IG02, which are upstream [. . . ] Also, the metrics for Tomcat01’s JVM memory usage (JVM-Memory_7778_JVM_Memory_HeapMemoryUsed) peaked at 1017709120.0, which is a very high value, suggesting memory issues. Answer 1. Tomcat01: High JVM memory usage and CPU idle spikes indicate memory exhaustion [. . . ] 2. Mysql02. 3. ServiceTest11.

(b) Misattributing a signal of another component Fig. 3. Two failed trajectories of Qwen3-4B. Blue marks evidence or intermediate conclusions that the final answer does not follow, and red marks the final answer and its evidence.

In all trajectories, the model examined the signals of two or more candidates and the propagation relationships between components. When selecting the cause, however, it repeatedly showed three behaviors that kept its final answer from following this analysis. • Behavior-1: Reliance on large anomaly scores. In 487 of the 640 trajectories (76 %), the model examined multiple candidates but gave a large anomaly score as the rationale for its final selection. In Fig. 3a, after checking Tomcat, IG, and MG, it selects Tomcat01 for its high JVM memory usage and repeated GC logs. • Behavior-2: Inconsistency with propagation analysis. In 433 trajectories (68 %), the final answer departed from the conclusion of the model’s own propagation analysis [39]. In Fig. 3a, the model suspects a problem upstream of the Tomcats but still selects the downstream Tomcat01. • Behavior-3: Misattributed signals. In 422 trajectories (66 %), the model attributed a signal of another component to its answer. In Fig. 3b, it cites the JVM heap signal of IG01 as “Tomcat01’s JVM memory usage” and selects Tomcat01. Effect of procedure injection. To test whether Table 1. Accuracy with a few-shot procedure. providing a procedure changes this selection, we Name: answers matching the example’s root appended to 𝑝 a few-shot example [4] in which cause. Family: same, ignoring instance numbers. DeepSeek-R1 [7] reasons step by step to the correct answer of another incident. When the example Condition Top-1 Name Family shares the root-cause component of the target, TopNo injection 16 % – – 1 (correct first answers) rises from 16 % to 56 %, but Same cause, names kept 56 % 56 % 81 % masking the component names in the example lowSame cause, names masked 21 % – – ers it to 21 % (Table 1). When the causes differ, 44 % Different cause, names kept 9 % 44 % 67 % of the answers copy the root-cause name of the example, and Top-1 falls to 9 %. The gain thus comes largely from the names rather than from the procedure. FoundRoot-14B [51], which learns the diagnostic procedure through SFT and RL, also

8

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

Offline

Historical Frequency Profiling Reference stats

Metrics μ, σ Traces mean latency

Fault-Free

Telemetry

Online OCE Services

Query + window W Telemetry Data

EOB r(s)

Metrics · Logs

Rare Common

Data Processing

Biased-Decision Correction Self-

sampling

Preference-pair

Construction Dfork Dpriv Dcredit

DPO

LoRA

adapter

Prompt (excerpt)

Query identify the root cause component and [...] reason. Metric frontend-0 / apm_pod_timeout · z = 1055.4 · r = 0.16%

cartservice / apm_service_timeout · z = 463.4 · r = 4.58% Log cartservice ×3: "Application is shutting down..." · r = 0.0% Trace cartservice: latency ×0.3 vs normal

SLM

RCA

Result

cartservice

/ pod fault

OCE

Fig. 4. System overview. Top: offline profiling and training. Bottom: online diagnosis of one incident.

relies on large anomaly scores (Behavior-1). On the inputs released by the authors, 72 % of its top-ranked answers are the component with the most anomalous metrics, and it is correct for 83 % of incidents when that component is the root cause but for none otherwise. Finding 2. The SLM examines candidates and analyzes propagation in every trajectory, but its final answer often does not follow this analysis. Providing the diagnostic procedure as a few-shot example or through training does not remove its reliance on large anomaly scores. The model should therefore be trained to select the root cause consistent with the evidence in the prompt while keeping its existing reasoning.

4

Design

We present an SLM-based RCA system that addresses Findings 1 and 2 by changing what the prompt states about each signal and how the SLM selects the root cause. Historical Frequency Profiling adds to the prompt how often each signal exceeds 𝜏 in the fault-free period, which the model cannot infer from the incident window alone (Finding 1). Biased-Decision Correction keeps the diagnostic procedure that the SLM already performs and trains only the decision part in order to let the SLM’s own reasoning lead to correct answers (Finding 2). Fig. 4 shows the architecture of our system. The offline path (top) runs once per target system. It profiles the fault-free period and trains a Low-Rank Adaptation (LoRA) [16] adapter with DPO on preference pairs built from the SLM’s own trajectories. The online path (bottom) builds the prompt 𝑝 for an incident window 𝑊 , attaching the EOB to each signal and ordering candidates by it, and the SLM with the adapter reads 𝑝 once to answer the component, fault type, and occurrence time. 4.1

Historical Frequency Profiling

Historical Frequency Profiling records, for each signal, how often it exceeds 𝜏 when no fault is present. Our system writes this frequency, the Expected Outlier Baseline (EOB), next to the anomaly score of each signal in the prompt and orders candidates by it. 4.1.1 Expected Outlier Baseline (EOB). The EOB 𝑟 (𝑠) of signal 𝑠 is the fraction of fault-free windows 𝑊 ′ ∈ Wnormal , defined as in §3.1, in which its anomaly score exceeds 𝜏:   𝑟 (𝑠) = ′ Pr 𝜙 (𝑠;𝑊 ′ ) > 𝜏 , (2) 𝑊 ∈ Wnormal

where 𝜙 (𝑠;𝑊 ′ ) is computed within 𝑊 ′ . Because 𝑟 (𝑠) is counted directly without assuming a distribution, it places KPIs with different 𝜎 and logs counted as entries on a single scale. The fault-free windows come from a profiling period long enough for 𝜇, 𝜎, and the EOB to become stable, over which the normal behavior of the system is assumed to stay similar.

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

9

For metrics, we use 𝜏 = 3 [10, 32] as in existing data processing but estimate 𝜇 and 𝜎 from all of Wnormal rather than from the few minutes before the incident window. For logs, 𝜏 = 0, so 𝑟 (𝑠) is the fraction of fault-free windows in which the template of 𝑠 appears. Traces have no EOB. Instead, the average latency of each service over Wnormal is stored as a baseline, and the prompt reports the ratio of the latency in the incident window to this baseline. Profiling needs the times of past incidents to identify the fault-free period but not their root causes. At diagnosis time, our system computes 𝜙 (𝑠) in 𝑊 with the profiled 𝜇 and 𝜎 and looks up 𝑟 (𝑠) for each signal that exceeds 𝜏. 4.1.2 Candidate Ordering and Prompt Rendering. At diagnosis time, the EOB serves two roles in the prompt. It is written next to the anomaly score of each signal, and it determines the order of the candidates. Prior work lists candidates without a defined order or by deviation magnitude [32, 51], which the anomaly score already conveys. Our system instead lists first the candidates whose anomalies are rare in the fault-free period. The prompt lists the following fields for each telemetry type, with the EOB written as a percentage (e.g., “fires in 0.39 % of normal windows”). • Metric: KPI name, anomaly score 𝜙 (written as 𝑧 max ), EOB, and the time the KPI first exceeded 𝜏. • Log: occurrence count and EOB of each template, per component. • Trace: span count, latency ratio to normal, and the first time the ratio exceeds 𝜂, per service. We set 𝜂 = 3, the same value as 𝜏 for metrics, so that a trace is flagged only when its latency deviates markedly from normal operation. Blocks for unavailable telemetry types are omitted. Algorithm 1 shows how the prompt 𝑝 is constructed. Lines 1–2 obtain the root-cause candidates Algorithm 1 EOB-Guided Input Construction E𝑊 from the metric signals flagged as anomalous Require: Query, 𝑊 , T𝑊 , 𝐺, F with fault-free statistics. Logs and traces do not add Require: Fault-free telemetry profiles candidates. Lines 3–8 compute, for each component Ensure: Model input 𝑝 1: 𝑀 ← metric observations with 𝜙 (𝑠) > 𝜏 𝑒, 𝑟 (𝑒) as the minimum EOB of its metric signals and 2: E𝑊 ← components in 𝑀 𝜙 max (𝑒) as the maximum anomaly score, and sort 3: for 𝑒 ∈ E𝑊 do its signals by EOB. Line 9 then sorts the candidates 4: 𝑀𝑒 ← observations of 𝑒 in 𝑀 in ascending order of 𝑟 (𝑒), breaking ties in descend- 5: 𝑟 (𝑒) ← min𝑠 ∈𝑀 𝑟 (𝑠) 𝑒 ing order of 𝜙 max (𝑒). Because 𝑟 (𝑒) is the EOB of the 6: 𝜙 max (𝑒) ← max𝑠 ∈𝑀𝑒 𝜙 (𝑠) rarest signal of a component, a component whose 7: Sort 𝑀𝑒 by EOB ↑ anomalous signals all exceed 𝜏 often in the fault-free 8: end for 9: Sort E𝑊 by 𝑟 (𝑒) ↑, then 𝜙 max (𝑒) ↓ period moves down even when its anomaly scores are large. Lines 10–11 sort logs by EOB and traces 10: 𝐿 ← logs sorted by EOB ↑ by latency ratio. Lines 12–15 write the sorted candi- 11: 𝑇 ← traces sorted by latency ratio ↓ dates and observations with render and append the 12: 𝑝 ← render(𝑊 , [(𝑒, 𝑀𝑒 )] 𝑒 ∈ E𝑊 , 𝐺) log and trace blocks, F when the query asks for the 13: Append 𝐿 and 𝑇 to 𝑝 14: Append F to 𝑝 if query requests fault type fault type, and the JSON output format. The SLM 15: Append requested JSON format to 𝑝 then reads 𝑝 once and selects the root cause from 16: return 𝑝 all three types of observations. 4.2

Biased-Decision Correction

Even with the EOB, the SLM often bases its final answer on a large anomaly score rather than on its own analysis (Finding 2). Biased-Decision Correction samples the SLM’s own reasoning, locates the points where it diverges into correct and incorrect answers, builds and verifies preference pairs from these points, and trains the decision that follows with DPO.

10

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee Response Generation Training prompt

SLM self-sampling

Identify the root-cause component and fault type.

Prefix:

Preference-pair Construction Decision fork

Rejected · incorrect

cartservice is

[…] l

apm_pod_timeout in

down, […] But the

ogs about shutting

frontend-0 […] maybe

highest z-score is in

that's a symptom of

frontend-0's

the cartservice issue.

apm_pod_timeout. […]

cartservice / pod fault

frontend-0 / pod fault

Dfor

Partial-credit pairs

Rescue trigger:

all 16 responses identify an incorrect component.

nput + privileged context

Original

idence in the cartservice section

response

I Ev

Element

Response A

Response B

Component

cartservice (+1)

cartservice (+1)

Fault type

pod fault (+1)

misconfig (0)

1+1=2

1+0=1

Total u

Re-sampling

Score gap Δu = 2 − 1 = 1 ≥ 0.5

Chosen · correct

Rejected · original

cartservice / pod fault

frontend-0 / pod fault

Remove hint before training

k

Same input + prefix

“Looking at frontend-0, there’s apm_pod_timeout […]”

cartservice has

shutting down, […] the

16 responses

Re-sampling

Cut at 30% · 45% · 60%

Privileged-context rescue

Chosen · correct […]

Prefix selection & review

Verification Check chosen reasoning against input

D tot

A → Chosen

Dpriv

Verified preference pairs

B → Rejected Dcre it d

DPO training Update LoRA adapter

Fig. 5. Preference-pair construction for Biased-Decision Correction. The chosen response of each pair is verified against the original input before DPO training.

4.2.1 Preference-Pair Construction. We call the model before training the reference policy 𝜋ref . All responses in the training pairs are generated by 𝜋 ref , because responses from the model itself are more effective for preference training than responses corrected by an external model [25]. A preference pair (𝑥, 𝑦 𝑤 , 𝑦𝑙 ) consists of a shared context 𝑥 and a chosen and a rejected continuation, 𝑦 𝑤 and 𝑦𝑙 . Fig. 5 illustrates the procedure, from self-sampling on the left to the three types of preference pairs and their verification on the right. Self-sampling and re-sampling. For each training incident, 𝜋 ref repeatedly generates reasoning and a final answer from its prompt 𝑝, forming trajectories. Datasets with fewer training incidents receive more trajectories per incident to raise the chance of observing different decisions. Among the component, fault type, and occurrence time asked by the query, one target element whose selection is to be corrected is chosen. The reasoning of each trajectory is cut at sentence boundaries at 30 %, 45 %, and 60 % of its length, so that prefixes end both during candidate examination and during answer composition. A reviewer LLM examines each prefix twice independently and discards those that have already fixed the answer for the target element or contain repetition, content absent from the input, or incomplete sentences. From each remaining prefix 𝑧 <𝜌 of a trajectory 𝑧 cut at ratio 𝜌, 𝜋ref generates 𝐽 continuations (𝐽 = 16 by default): 𝑥 = 𝑝 ⊕ 𝑧 <𝜌 ,

𝑦 (1) , . . . , 𝑦 (𝐽 ) ∼ 𝜋ref (· | 𝑥).

(3)

Decision fork (Dfork ). When the continuations of the same context 𝑥 include both correct and incorrect answers for the target element, we call 𝑥 a decision fork. Each pair of a correct and an incorrect continuation from 𝑥 is added to Dfork . In the decision fork example of Fig. 5, the target element is the component, and both continuations examine the termination log of cartservice. The rejected continuation, however, selects frontend-0 based on its high 𝑧 max , whereas the chosen continuation links the termination log to a pod fault of cartservice and interprets the timeout of frontend-0 as its effect. This pair, which selects different root causes from the same prefix, is added to Dfork . Privileged-context rescue (Dpriv ). When no continuation answers the target element correctly even after re-sampling, a pair cannot be formed. We then append privileged context, auxiliary information about the correct answer, to the prompt and re-sample from the same prefix. We first give only the component that holds the evidence, or an approximate time of the fault for queries that ask for the time, and add the correct label only if this still yields no correct continuation. The

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

11

resulting correct continuation becomes the chosen response and is paired with an original incorrect continuation on the original context 𝑥. In the rescue example of Fig. 5, cartservice / pod fault obtained this way is paired with the original frontend-0 / pod fault. The privileged context thus serves only to obtain the chosen response and is absent during training. Unlike STaR [56], which uses such rationalized solutions for SFT, we pair them with the original incorrect answers, discard continuations that mention the privileged context, and check the rest for grounding in the original input. Partial-credit pairs (Dcredit ). Continuations with the same correctness for the target element can still differ in the other requested elements. We therefore score each continuation by 𝑢, the sum over the requested elements of 1 point for a correct component, 1 point for a correct fault type, and 1 point for an occurrence time within ±60 seconds (0.5 points within ±300 seconds). We pair the top two continuations of each prefix by 𝑢 with the bottom two and keep the pairs with Δ𝑢 ≥ 0.5 whose continuations agree on the correctness of the target element, because the other pairs are already covered by Dfork . In the partial-credit example of Fig. 5, both continuations identify cartservice, but only the one that answers pod fault identifies the fault type, so the response with 𝑢 = 2 is chosen over the one with 𝑢 = 1. The score 𝑢 only selects pairs and does not weight the loss. Verification. A correct final answer may come with an explanation that is not grounded in the input. Therefore, the chosen responses of the three sets are reviewed twice independently by an LLM. Each review checks whether a response misattributes the observations of another component, presents values or times absent from the input, or misreads a metric’s value or meaning. If either of the two judgments reports a problem, the preference pair is removed. 4.2.2 DPO Training. Biased-Decision Correction trains the target model 𝜋𝜃 with DPO [38] on the preference pairs that pass verification, Dtotal = Dfork ∪ Dpriv ∪ Dcredit . Without a separate reward model, DPO minimizes the following loss so that the chosen response is preferred over the rejected response in the same context: h  𝜋𝜃 (𝑦 𝑤 | 𝑥) 𝜋𝜃 (𝑦𝑙 | 𝑥)  i , (4) LDPO = −E (𝑥,𝑦𝑤 ,𝑦𝑙 )∼Dtotal log sigmoid 𝛽 log − 𝛽 log 𝜋ref (𝑦 𝑤 | 𝑥) 𝜋 ref (𝑦𝑙 | 𝑥) where 𝛽 controls the deviation from 𝜋ref . Because the input and the prefix serve only as the condition 𝑥, the loss is computed only on the tokens of the continuations. The training signal thus compares which reasoning and answer should follow the prefix rather than teaching the model to reproduce the prefix. Dtotal is constructed before training, no new responses are generated during training, and every pair has the same weight. Training uses LoRA [16], which freezes the weights of the base model and updates only the adapter. We train one adapter per dataset on all projections of the attention and MLP layers (rank 16, 𝛼 = 32, dropout 0.05), with 𝛽 = 0.1, a learning rate of 5 × 10−6 under a cosine schedule with 10 % warmup, 4 epochs, and an effective batch size of 8. The reference policy is the base model without the adapter, and the prompts for both training and evaluation are constructed by Algorithm 1. 5

Evaluation

We evaluate our system by answering the following five research questions. • RQ1 (Effectiveness): How accurately does our system perform RCA compared with existing RCA systems that use commercial or fine-tuned models? (§5.2) • RQ2 (Ablation Study): How much do the EOB and the three types of preference pairs each contribute to accuracy? (§5.3) • RQ3 (Generalizability): Does an adapter trained on one dataset generalize, without further fine-tuning, to held-out datasets and systems? (§5.4)

12

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

• RQ4 (Model, Model Size, and Quantization): How do the base model, its size, and quantization affect the accuracy and deployment efficiency of our system? (§5.5) • RQ5 (Deployment Overhead): How much fault-free history and offline computation does deploying our system on a target system require? (§5.6) 5.1

Experimental Setup

Table 2 summarizes the datasets and their training Table 2. Datasets and their training and and evaluation incidents. Telecom, Bank, and Market evaluation incidents. of OpenRCA [52] are collected from a telecommunication database system, a banking system, and the eDataset System Train Eval. commerce application HipsterShop, respectively, and AIOps 2025 [35] is collected by injecting faults into Training and evaluation within each system Telecom [52] Telecom service 30 21 HipsterShop deployed on Kubernetes. On these four Bank [52] Banking service 80 56 datasets, we train an adapter on the training incidents Market [52] HipsterShop 88 60 of each system and measure accuracy on its evaluation AIOps 2025 [35] HipsterShop 241 159 incidents by randomly splitting the incidents of each Evaluation on held-out datasets dataset into 60 % for training and 40 % for evaluation. DejaVu-A [28] Major ISP system – 63 – 90 We evaluate generalization across systems on DejaVu- DejaVu-B [28] Major ISP system – 10 A and DejaVu-B [28], Eadro-SN [26], and GAIA [6], the Eadro-SN [26] SocialNetwork MicroSS – 255 four held-out test datasets of FoundRoot, none of which GAIA [6] A dash marks datasets not used for training. FoundRoot or our system uses for training. The GAIA test set released by FoundRoot labels only the root-cause component, so only the component is evaluated on it. 5.1.2 Evaluation Metrics. Following OpenRCA [52], we score only the component, fault type, and occurrence time that the query requests. The component is compared at the service or instance level specified by the label, and the fault type must match the label in the list of fault types of the benchmark. The occurrence time is correct if it is within 60 seconds of the label. For queries that ask about multiple faults, predicted faults are matched to labeled faults before scoring. Exact is the ratio of responses that correctly answer all elements the query requests, which OpenRCA calls Correct [52]. Partial is the average fraction of requested elements answered correctly per response. This definition differs from the Partial of OpsAgent [32], which counts a response as partially correct if it answers at least one requested element. Component, Fault Type, and Time score each element separately. Among the queries that request the element, each is the fraction of responses that answer it correctly, where a response to a multi-fault query counts as correct if it matches at least one labeled value. 5.1.3 Baselines. We compare our system with three groups of baselines. • Statistical RCA. Nezha [54] localizes the root cause from changes in event patterns constructed from metrics, logs, and traces. KPIRoot [13] ranks KPIs from metric time series by combining the symbolic sequence similarity between the alert KPI and candidate KPIs with Granger causality. KPIRoot+ [14] extends KPIRoot by detecting anomalous intervals with STL decomposition and a U-Net autoencoder representing change trends with iSAX [43], a symbolic time-series encoding. • LLM-based RCA without fine-tuning. CoT [50] prompts the off-the-shelf Qwen3-4B to reason step by step over the prompt built by the data processing of OpsAgent [32]. ReAct [53] lets the same model alternate, for up to 10 turns, between reasoning and calls to the trace, log, and metric query tools released with ThinkFL [60]. RCA-Agent [52], designed for commercial

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

13

models, investigates telemetry with a Controller that plans the analysis and an Executor that runs generated Python code. We run it with GPT-5-mini [34] and Solar Pro 4 [46]. • RCA with fine-tuned models. OpsAgent-14B [32] trains expert agents with PPO and has them cross-review their diagnoses. FoundRoot-14B [51] trains structured deep thinking with SFT and RL to rank components from metrics. ThinkFL-8B [60] fine-tunes Llama with SFT on tool-calling conversations and then with GRPO to query traces and metrics from the entry span of a trace. We run Nezha with the authors’ implementation after reproducing its reported performance, and KPIRoot with its public code with minor fixes. We implement KPIRoot+ by adding the three changes described in its paper to this code, and both use the service-level objective metric of each system as the alert KPI [13, 14]. For RCA-Agent, we score one run of the public code per incident [22], and for OpsAgent, the authors’ released results on our evaluation incidents. FoundRoot and ThinkFL use their released checkpoints (and tools for ThinkFL), with fault type and occurrence time fields added to their prompts and answer formats because both output only components. 5.1.4 Implementation. The default backbone of our system is Qwen3-4B [45] in thinking mode. Historical Frequency Profiling uses the entire fault-free period of each dataset, and §5.6 evaluates shorter profiling periods. For self-sampling, the reference policy generates trajectories at temperature 0.6, 32 per training incident for Telecom, Bank, and Market and 8 per training incident for AIOps 2025. Claude Opus 5 [3] serves as the reviewer LLM for prefix review and verification. The reviewer is used only offline to build training pairs from benchmark data, and online diagnosis sends no telemetry outside the organization. In production, a self-hosted model can take this role. Biased-Decision Correction trains the LoRA adapter [16] with the DPOTrainer of TRL [18] under the settings in §4.2.2, with a maximum sequence length of 12,288 tokens (18,432 for Market) so that no pair is truncated. Qwen3-4B is trained on a single NVIDIA L40S (48 GB), and the 14B model in RQ4 on a single NVIDIA A100 (80 GB). Inference runs on vLLM [24] with temperature 0.6, top-p 0.95, top-k 20, and a maximum generation length of 6,144 tokens. We generate eight responses per incident, score each independently, and report their average (avg@8) [23, 32, 36, 37]. Off-the-shelf denotes the same base model with the same generation settings but without the LoRA adapter. Unless noted otherwise, it receives the EOB prompt. 5.2

RQ1: Effectiveness

Against all baselines on the four datasets (Table 3), our system achieves the highest Partial everywhere and the highest Exact on all but Market, where it ties with OpsAgent-14B. The statistical methods do not output the fault types or occurrence times of the benchmarks, so they cannot complete such queries, and the Exact of Nezha and the KPIRoot family stays below 2 % on all four datasets. Statistical EOB, which returns the first candidate in the EOB-ordered list, achieves higher Partial than the other statistical methods on Telecom, Bank, and AIOps 2025, but its Exact is nonzero only on Bank, where some queries request only the component. The gap from our system, which reads the same prompt to identify all requested elements, shows the role of the model in completing the diagnosis from this evidence. Compared with LLM-based methods without fine-tuning, our system achieves higher Exact and Partial on all datasets than CoT and ReAct, which use the same Qwen3-4B, and than RCA-Agent with GPT-5-mini and Solar Pro 4. Market is the most challenging dataset, where no method exceeds 11.9 % Exact. The difficulty concentrates on two-fault queries, which make up 45 % of Market (27 of 60) but at most 11 % of the other datasets. Every baseline answers at most 4.0 % of these queries correctly, whereas our system answers 9.7 %. On single-fault queries, OpsAgent-14B leads instead (17.6 % vs. 13.6 %), so the two methods tie on Market overall.

14

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

Table 3. Accuracy on four datasets (%). Exact: ratio of responses that answer all requested elements correctly. Partial: average fraction of requested elements answered correctly per response. Method

Telecom

Bank

Market

AIOps 2025

Exact Partial

Exact Partial

Exact Partial

Exact Partial

0.0 0.0 0.0 0.0

2.4 2.4 6.3 10.3

1.8 1.8 0.0 7.1

4.8 1.8 0.9 17.7

0.0 1.7 1.7 0.0

1.2 3.6 2.5 0.3

0.0 0.0 0.0 0.0

4.4 2.5 4.2 15.3

6.5 3.0 14.3 4.8

15.0 10.8 23.8 15.9

2.5 9.6 7.1 12.5

7.9 15.2 12.9 21.4

3.8 4.2 5.0 1.7

12.9 13.6 17.4 9.7

0.5 2.3 2.5 8.2

20.3 19.1 22.3 26.1

30.0 0.0 39.3

37.5 0.0 55.3

18.5 0.0 48.7

27.6 0.0 55.5

11.9 0.0 11.9

23.0 1.8 24.3

N/A 0.0 21.1

N/A 8.0 52.4

Statistical RCA Nezha [54]† KPIRoot [13] KPIRoot+ [14] Statistical EOB (Ours) LLM-based RCA without fine-tuning CoT (Qwen3-4B) [50] ReAct (Qwen3-4B) [53] RCA-Agent [52] (GPT-5-mini [34])‡ RCA-Agent [52] (Solar Pro 4 [46]) RCA with fine-tuned models OpsAgent-14B [32]§ ThinkFL-8B [60] ∥ Ours-4B † Outputs not mapped to benchmark fault types.

‡ Public code, one run per incident [22].

§ Authors’ result files (Qwen2.5-14B); no AIOps 2025.

∥ Llama 3 8B trained on HipsterShop (Market, AIOps 2025).

OpsAgent-14B, in which expert agents trained with PPO diagnose and review each other’s results [32], is the strongest baseline on Telecom, Bank, and Market. Our system outperforms it on both metrics on Telecom and Bank and achieves a higher Partial on Market. ThinkFL is trained on HipsterShop [60], the system of Market and AIOps 2025, yet its Exact is 0 on both. Its Partial comes only from the ranked components, and none of its responses gives the fault type or occurrence time, even though its prompt and answer format request them. Answer to RQ1. With a 4B model and without cross-agent review or repeated tool calls on raw telemetry, our system matches or exceeds every baseline on both metrics across all four datasets, including the 14B OpsAgent and RCA-Agent with commercial models.

5.3

RQ2: Ablation Study

Accuracy (%)

To see how each component of our system conFault type Component Time tributes, Fig. 6 adds the components of our system 75 cumulatively from left to right on the x-axis and 58.4 55.8 50.6 47.8 shows the Component, Fault Type, and Time scores 50 51.1 48.8 on the four evaluation splits. The leftmost step is 43.8 39.2 41.1 38.8 the off-the-shelf Qwen3-4B on the prompt of ex32.6 25 20.8 26.0 19.6 isting data processing. +EOB replaces this prompt 13.3 with the EOB prompt, and each later step adds one 0 lf it B iv pair type and trains a new adapter from the base rk she +EO + fo + pr  cred e + h model on all pairs added so far. t f Of +EOB. Without fine-tuning, the EOB prompt raises all three elements. Component rises because the Fig. 6. Per-element accuracy as components root cause is listed first more often. On Bank, the are added cumulatively (avg@8, %). first candidate in the EOB prompt is the root cause for two thirds of incidents, whereas the component with the largest anomaly score in the existing prompt is the root cause for only 8 %. Time rises the most because the EOB prompt states, for each

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

15

candidate, the first time its rarest signal exceeds 𝜏, whereas answering the first timestamp in the existing prompt is correct for fewer than 20 % of incidents. Fault Type stays the lowest of the three because the prompt does not point to it, and it requires diagnosing what the signals of the chosen component indicate. +Dfork . These pairs make the model reach, in more of its responses, a correct decision that it previously reached in only some of them. Because accuracy is averaged over eight responses per incident, most of the gain at this step on every dataset comes from such incidents. +Dpriv . Contexts in which no continuation is correct give Dfork no chosen response, and Dpriv supplies one through privileged context, which is absent from the training input. These pairs show the model a line of reasoning from the evidence in its own prompt to the full answer, and all three elements rise at this step. On Bank, for example, Exact rises from 33.0 % to 47.1 %. +Dcredit . Dfork and Dpriv separate correct answers from incorrect ones but do not guide a partly correct response, such as one with the right component, toward the remaining elements. Dcredit fills this gap by preferring, between two responses that share part of the answer, the one that answers more requested elements correctly, and all three elements rise at this step on average, although Component on Bank and Fault Type on Market fall by 1.5 and 3.9 points. Answer to RQ2. Every component raises accuracy in a different way. The EOB prompt alone ranks the root cause higher and shows when each anomaly begins. Among the pairs, Dfork makes correct decisions consistent, Dpriv covers incidents never answered correctly, and Dcredit completes partly correct responses.

5.4

RQ3: Generalizability

To evaluate whether our system generalizes to systems unseen in training, we apply the Table 4. Accuracy on the held-out test datasets of FoundRoot (%). Each method uses its own data proadapter trained on the Dtotal pairs of Bank to cessing. Comp.: Component. Type: Fault Type. the four held-out datasets without further finetuning and compare it with FoundRoot-14B in Dataset Method Exact Partial Comp. Type Time Table 4. Our system builds its prompt with AlDejaVu-A FoundRoot-14B 1.6 22.2 41.3 12.7 12.7 gorithm 1 and computes the EOB from the fault- (63) Ours-4B 10.7 50.9 44.2 28.8 79.6 free period of each target system, while Found- DejaVu-B FoundRoot-14B 0.0 21.9 38.9 12.2 14.4 Ours-4B 6.4 38.2 40.4 25.6 48.8 Root builds its input for the same incident win- (90) dows with its own data processing. 70.0 70.0 60.0 Eadro-SN FoundRoot-14B 40.0 66.7 Ours-4B 70.0 76.7 70.0 71.3 88.8 On DejaVu-A, DejaVu-B, and Eadro-SN, the (10) FoundRoot-14B – – 88.6 – – two methods differ more in fault type and GAIA (255) Ours-4B – – 68.6 – – time accuracy than in component accuracy. On (n): number of incidents. –: GAIA has no fault type or time labels. Eadro-SN, both identify the root-cause component in 70.0 % of the incidents, but our system identifies the occurrence time more often (88.8 % vs. 60.0 %) and raises Exact from 40.0 % to 70.0 %. On DejaVu-A and DejaVu-B, higher Fault Type and Time raise Partial from 22.2 % to 50.9 % and from 21.9 % to 38.2 %, although Exact stays at most 10.7 % because Component and Fault Type remain below 45 % and 30 %. This gap in fault type and time accuracy reflects the design of our system, whose prompt states when each signal first exceeds 𝜏 and whose preference pairs reward every requested element. FoundRoot instead learns only to rank root-cause components [51], a strength that shows on GAIA, which evaluates only the component (88.6 % vs. 68.6 %). On the other three datasets, our system reaches component accuracy comparable to FoundRoot’s with a 4B adapter trained on one system while also identifying the fault type and time. Bank and the target systems differ in component names and fault type notation but share phenomena such as CPU load and network faults. Our system receives the candidates and fault

16

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

types of each target system, and the EOB is a ratio to each system’s own fault-free period, so its values mean the same across systems. The adapter can therefore reuse the judgment learned on Bank to select the root cause from observations. Answer to RQ3. Trained on Bank, the 4B adapter transfers to unseen systems without further fine-tuning. On three of them, it matches FoundRoot-14B in component accuracy and exceeds it in Exact and Partial by identifying the fault type and time, while FoundRoot-14B leads on GAIA, which evaluates only components.

5.5

RQ4: Model, Model Size, and Quantization

Seconds per incident

Exact (%)

To compare deployment configurations on a Off-the-shelf Ours Ours single GPU, Fig. 7 evaluates Off-the-shelf and bf16 bf16 Int4 40 80 Qwen3 Gemma 4 Ours in bf16 and Ours in GPTQ Int4 at four 30 60 Qwen3 sizes, and the same three configurations on Gemma 4 E4B [12], a model of an20 40 other family. Exact is averaged over the three 10 20 OpenRCA datasets, and latency is measured on 0 0 one NVIDIA L40S. All configurations use the 0.6B 1.7B 4B 14B E4B same EOB prompt, and our system trains on preference pairs from the trajectories of each Fig. 7. Exact (bars) and latency per incident model. For Int4, we merge the LoRA adapter into (lines) by model and size. All models, including Offthe base model in float32 and apply GPTQ [11] the-shelf, receive the EOB prompt. Error bars show W4A16 with group size 128, calibrated on train- the standard deviation over eight runs with one response per incident. Int4: GPTQ W4A16. ing prompts. Our system raises the mean Exact of Qwen3-4B from 20 % to 33 %, surpassing the off-the-shelf 14B model at 26 % and approaching the fine-tuned 14B model at 34 % with 7.8 instead of 28.0 GiB of weights. The 0.6B and 1.7B models reach only 19–20 % after training. Training also raises Gemma 4 E4B from 18 % to 30 %. Gemma 4 E4B answers faster than Qwen3-4B (32.7 vs. 44.0 seconds per incident in bf16) because it generates fewer tokens per response, but its Exact is lower. GPTQ Int4 reduces weight size and latency for every evaluated model. For 4B, Exact decreases from 33 % to 31 %, while the weights shrink from 7.8 to 2.5 GiB and latency falls from 44.0 to 20.0 seconds per incident. For 14B, Int4 keeps the bf16 Exact with 9.3 GiB of weights and cuts latency from 55.9 to 25.3 seconds, below that of bf16 4B, consistent with prior reports that GPTQ 4-bit can change accuracy in either direction [9, 27]. 14B Int4 is thus an option when accuracy matters more than the smaller weights of 4B Int4. Answer to RQ4. Biased-Decision Correction improves accuracy at every evaluated model size and on a model of another family. Our system with 4B surpasses the off-the-shelf 14B model and approaches the fine-tuned 14B model with smaller weights and lower latency. Int4 further reduces weight size and latency, while preserving bf16 accuracy at 14B.

5.6

RQ5: Deployment Overhead

Deploying our approach on a new system requires fault-free history and one offline run, so we measure how much fault-free window it needs and how long training takes. Fig. 8 shows how Exact and Partial change with a shorter fault-free period. For each incident, we keep the fault-free windows nearest the incident, before or after it, rerun Historical Frequency Profiling on these windows, and evaluate our 4B model with the same adapter. The horizontal axis is the length of the kept windows on a log scale, and its right end is the entire fault-free period, which is 80, 211, and 18 hours for Telecom, Bank, and Market. A negative value on the vertical axis means a loss.

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

17

Change in % (pp)

Averaged over the three datasets, shortening Partial Exact Telecom Bank Market the profiling period to about half a day lowers 20 Exact by at most 3.4 percentage points. The 10 0 largest loss is 29.1 points on Bank with 1.7 hours, −10 where only 10 fault-free windows remain per in−20 cident instead of 1,269. With 𝜇 and 𝜎 estimated −30 from these windows, more metrics are flagged 1h 1d 2h 1 d 8.8 d 2h 18 h as anomalous in 𝑊 (123.1 instead of 53.6 per inFault-free period used for profiling cident), and 27.7 % of them receive an EOB of 0 instead of 0.5 %, so they appear as rare as metrics Fig. 8. Accuracy change by profiling period length. Relative to the entire fault-free period. Shadthat exceed 𝜏 only during failures. For the four datasets, the offline path of our ing shows ±1 standard deviation over eight runs with system with Qwen3-4B takes 70.3 hours in total. one response per incident. Historical Frequency Profiling takes 0.4 hours on one CPU, preference-pair construction takes 13.2 hours on one L40S, and DPO training on the resulting 5,688 pairs takes 56.7 hours on one L40S. The pair-construction time is the inference time of self-sampling and re-sampling, estimated from the measured time per response. Answer to RQ5. Using more of the fault-free period helps, but our system does not need the entire period. Averaged over the three datasets, Exact decreases by at most 3.4 points when the profiling period is shortened to about half a day. Building our system with Qwen3-4B for the four datasets takes 70.3 hours, mostly for DPO training, on a single 48 GB GPU rather than a multi-GPU cluster.

5.7

Case Study

Fig. 9 compares the cause selection of the off-the-shelf Qwen3-4B and our system on the same EOB input. The incident, qa_229 of AIOps 2025, is a fault injected at the node level into the Kubernetes cluster that runs HipsterShop (§5.1). The query gives the 30-minute window from 18:00 to 18:30 on June 17, 2025, states that one failure occurred in it, and asks for the root-cause component, the fault type, and the occurrence time, with a list of fault types that includes node fault, pod fault, and network attack. The labels are the node aiops-k8s-03, node fault, and 18:10:43. The prompt built by Algorithm 1 lists 28 candidates in ascending order of EOB with the fields of §4.1.2. The first candidate is the pod checkoutservice-1: its request and response counts exceeded 𝜏 at 18:14 and do so in 0.00 % of the fault-free windows, and its maximum response time has an anomaly score of 362. The node is second. Its available memory (infra_node_node_memory_MemAvailable_bytes) exceeded 𝜏 at 18:11 with an anomaly score of 3.5 and an EOB of 0.00 %, and its memory usage rate, disk read time, and disk read bytes exceeded 𝜏 in the same minute with EOBs between 0.05 and 1.01 %. frontend is the 13th candidate, with an EOB of 1.92 % for its rarest metric. The log block holds two entries of frontend, “failed to complete the order: rpc error: code = Internal” and the same with “Unavailable”, each written once in the window and in 0.0 % of the fault-free windows; they record that an order failed because a call to a backend service returned an error. The other templates are startup logs of adservice and transaction logs of paymentservice, with EOBs of 1.4 % and above. The trace block gives the average latency of each service in the window relative to its fault-free average (§4.1.1): recommendationservice rose to 5.5 times and first exceeded three times at 18:11, whereas frontend stayed at 0.8 times. The node thus has the rarest signal at the earliest time but neither the first position in the list nor the largest anomaly score, and the only evidence with an EOB of 0.0 % that names frontend is its two error logs. The two responses perform the same analysis. Both examine the memory signal of the node, the error logs of frontend, and the latency ratios, and both note that the latency of frontend is 0.8

18

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

Case: AIOps 2025 qa_229. Ground truth: aiops-k8s-03 / node fault at 18:11. Observed in both responses: aiops-k8s-03 memory signal first crossed 𝑧 ≥ 3 at 18:11 (fires in 0.00 % of normal windows); frontend log “failed to complete the order: rpc error” (0.0 %); recommendationservice latency ×5.5, frontend latency ×0.8. (a) Off-the-shelf

(b) Ours

Alternatively, the aiops-k8s-03 component has a node memory issue at 18:11:00, which is 0.00% normal. [. . . ] However, the logs are from the frontend. [. . . ] So the component is frontend, reason is network attack, occurrence time 18:11:00.

However, the frontend’s latency is only x0.8, which is lower than normal. So maybe the frontend isn’t the issue. [. . . ] The aiops-k8s-03’s first signal is infra_node_node_memory_MemAvailable_bytes crossing z≥3 at 18:11:00 with 0.00% normal windows. [. . . ] Maybe the node fault in aiops-k8s-03 caused the recommendationservice to have high latency. Answer: aiops-k8s-03 / node fault correct

Answer: frontend / network attack

wrong

Fig. 9. Root-cause selection before and after training on the same input. One response per condition on the same input.

times its baseline. They diverge only at the decision: the off-the-shelf response lets the location of the error logs decide, reads the ratio of 0.8 as high, and selects frontend, the pattern of Finding 2 in which the final selection departs from the model’s own analysis. The trained response weighs the same observations differently: the normal latency of frontend rules it out, and the latency of recommendationservice is attributed to the node fault. This is the decision that Biased-Decision Correction targets. Its pairs contrast continuations that share the analysis and differ in the selection (§4.2.1), and the result is visible here: the off-the-shelf model names the node in the reasoning of seven of its eight responses but selects it in one, whereas the trained model names and selects it in all eight. 6

Related Work

Table 5 compares our system with exTable 5. Properties of RCA approaches. isting approaches along five properties. Statistical methods reflect norNormal Decision- SelfSemantic Full behavior level hosted mal behavior but rank signals by sta- Category reasoning diagnosis context training LLM tistical patterns rather than by what – – the signals mean, and they do not an- Statistical [37, 54, 57] Prompted LLM [5, 44, 52] – swer the fault type and time. LLM- Fine-tuned SLM [51, 60] based methods reason about what sig- Multi-agent SLM [32] nals mean, but most compare them Reasoning preference – – – – learning [20, 25, 30] with the minutes around the incident window. With prompting alone, self- Ours full, partial, no support, and – not applicable. hosted models are far less accurate than commercial ones, and fine-tuned methods apply the training signal to whole responses. Preference learning for general reasoning applies the signal to the decision where reasoning diverges but does not address RCA. Our system combines these properties in a self-hosted 4B model, giving it how often each signal is flagged as anomalous without faults and training only the decision where its own reasoning diverges. Anomaly flagging in RCA. Existing methods take a baseline from the minutes around the incident window [32, 51] or from the fault-free period [19, 37, 44, 57] and give the model or the ranker how far each signal deviates from it. Nezha and SpecRCA also use how often something occurs without faults, but only to filter alerts or to adjust a score [54, 59]. From the same fault-free

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

19

period, our system counts how often each signal is flagged as anomalous without faults and gives this frequency to the model as the EOB next to the anomaly score. Even when the anomaly score alone cannot separate a routine deviation from a failure, the model can thus recognize a signal that is flagged as anomalous in the fault-free period as well. These baselines also use telemetry after the incident, from the minutes following the incident window [32, 51] to the normal interval after each incident [44], and our system likewise profiles fault-free windows on both sides of each incident. Learning RCA reasoning. The RCA training in §2.2.3 applies the training signal to whole responses, and SpecRCA fine-tunes a 3B verifier on the responses of a teacher LLM while a large LLM synthesizes the final diagnosis [59]. Our system instead targets final selections that depart from evidence already in the input (Finding 2). It contrasts continuations that select different causes after the model’s own common prefix and trains only this selection with DPO [38], without a teacher model or a larger model online. Preference construction for reasoning. Process supervision scores each reasoning step [29], and Math-Shepherd [47] and cDPO [30] obtain such scores automatically by sampling continuations from the prefix of each step or token and measuring how often they reach the correct answer. Step-DPO [25] and Self-Explore [20] contrast correct and incorrect steps that share the prefix before the first error. Our system also samples continuations from prefixes of its own reasoning, but it sets the preference by the per-element answers of the RCA query, which also yields pairs between partially correct answers, and verifies each chosen response against the input. STaR gives the answer as a hint to obtain rationales for failed problems and fine-tunes on them without the hint [56]. The privileged-context pairs of our system likewise drop the hint but pair the obtained continuation with the original incorrect one for DPO. 7

Limitations and Threats to Validity

Fault-free profiles. As in prior work that uses telemetry from both before and after each incident [44, 51], our evaluation profiles fault-free windows on both sides of each incident, whereas a deployment would profile only earlier telemetry. RQ5 indicates that about half a day of fault-free windows near an incident suffices, and profiling only earlier windows remains future work. The profile may also lose relevance after the workload or service configuration changes, and a signal that is frequent during normal operation can still be related to a failure. Quality of preference pairs. Biased-Decision Correction depends on the training incidents and the quality of the preference pairs. A response with a correct final answer may still contain an incorrect causal explanation, and the reviewer LLM that checks it may misjudge it. The reviewer is a commercial model used only offline, and we did not evaluate a self-hosted model in this role. Implementation of baselines. The main threat to internal validity lies in the implementation of the baselines. KPIRoot+ is reimplemented from its paper due to the lack of public code, and KPIRoot is run from its public code with minor fixes. For all other methods, we use the official implementations, released checkpoints, or the authors’ released results. Although we followed the described algorithms as closely as possible, potential discrepancies may exist. Scope of evaluation. The trajectory analysis targets Bank and Qwen3-4B, and the model comparison covers only the Qwen3 family and Gemma 4 E4B. The effects on other model families and on production environments that change over time, as well as on the investigation and recovery time of operators, require further validation. 8

Conclusion

In this work, we presented an SLM-based RCA system motivated by the gap between performing diagnostic analysis and selecting the root cause. Our empirical study showed that a model can examine candidates and propagation relationships yet still favor large anomaly scores in its final

20

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

decision, while its input omits whether those anomalies are common during normal operation. Our system addresses these limitations by providing normal-behavior context and training the model on its own responses that reach different conclusions from the same initial analysis. With identical input, this targeted correction enables a 4B model to outperform an off-the-shelf 14B model. Using a single inference call per incident, our system achieves the highest exact-match accuracy on three of four evaluation datasets and ties with the best baseline on the fourth. These results highlight evidence interpretation and final root-cause selection as important targets for improving SLM-based RCA, alongside the diagnostic procedures that models follow. References [1] Toufique Ahmed, Supriyo Ghosh, Chetan Bansal, Thomas Zimmermann, Xuchao Zhang, and Saravan Rajmohan. 2023. Recommending Root-Cause and Mitigation Steps for Cloud Incidents Using Large Language Models. In Proceedings of the 45th International Conference on Software Engineering (Melbourne, Victoria, Australia) (ICSE ’23). IEEE Press, 1737–1749. doi:10.1109/ICSE48619.2023.00149 [2] Amazon Web Services. 2026. Amazon Bedrock. https://aws.amazon.com/bedrock/. Accessed: 2026-10-02. [3] Anthropic. 2026. Claude. https://www.anthropic.com/claude. Accessed: 2026-10-02. [4] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS ’20). Curran Associates Inc., Red Hook, NY, USA, Article 159, 25 pages. [5] Yinfang Chen, Huaibing Xie, Minghua Ma, Yu Kang, Xin Gao, Liu Shi, Yunjie Cao, Xuedong Gao, Hao Fan, Ming Wen, Jun Zeng, Supriyo Ghosh, Xuchao Zhang, Chaoyun Zhang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Tianyin Xu. 2024. Automatic Root Cause Analysis via Large Language Models for Cloud Incidents. In Proceedings of the Nineteenth European Conference on Computer Systems (Athens, Greece) (EuroSys ’24). Association for Computing Machinery, New York, NY, USA, 674–688. doi:10.1145/3627703.3629553 [6] CloudWise. 2021. GAIA Dataset. https://github.com/CloudWise-OpenSource/GAIA-DataSet [7] DeepSeek-AI. 2025. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature 645, 8081 (Sept. 2025), 633–638. doi:10.1038/s41586-025-09422-z [8] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. A Survey on In-context Learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 1107–1128. doi:10.18653/v1/2024.emnlp-main.64 [9] Abhinav Dutta, Sanjeev Krishnan, Nipun Kwatra, and Ramachandran Ramjee. 2024. Accuracy is Not All You Need. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 124347–124390. doi:10.52202/079017-3950 [10] Aoyang Fang, Songhan Zhang, Yifan Yang, Haotong Wu, Junjielong Xu, Xuyang Wang, Rui Wang, Manyi Wang, Qisheng Lu, and Pinjia He. 2025. Rethinking the Evaluation of Microservice RCA with a Fault Propagation-Aware Benchmark. arXiv:2510.04711 [cs.SE] https://arxiv.org/abs/2510.04711 [11] Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. GPTQ: Accurate Quantization for Generative Pre-trained Transformers. In The Eleventh International Conference on Learning Representations. https://openreview. net/forum?id=tcbBPnfwxS [12] Gemma Team. 2026. Gemma 4 Technical Report. arXiv:2607.02770 [cs.CL] https://arxiv.org/abs/2607.02770 [13] Wenwei Gu, Xinying Sun, Jinyang Liu, Yintong Huo, Zhuangbin Chen, Jianping Zhang, Jiazhen Gu, Yongqiang Yang, and Michael R. Lyu. 2024. KPIRoot: Efficient Monitoring Metric-based Root Cause Localization in Large-scale Cloud Systems. In 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 403–414. doi:10.1109/ISSRE62328.2024.00046 [14] Wenwei Gu, Renyi Zhong, Guangba Yu, Xinying Sun, Jinyang Liu, Yintong Huo, Zhuangbin Chen, Jianping Zhang, Jiazhen Gu, Yongqiang Yang, and Michael R. Lyu. 2025. KPIRoot+: An efficient integrated framework for anomaly detection and root cause analysis in large-scale cloud systems. Empirical Softw. Engg. 31, 2 (Dec. 2025), 37 pages. doi:10.1007/s10664-025-10769-0 [15] Pinjia He, Jieming Zhu, Zibin Zheng, and Michael R. Lyu. 2017. Drain: An Online Log Parsing Approach with Fixed Depth Tree. In 2017 IEEE International Conference on Web Services (ICWS). 33–40. doi:10.1109/ICWS.2017.13

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

21

[16] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. https://openreview.net/forum?id=nZeVKeeFYf9 [17] Junjie Huang, Zhihan Jiang, Zhuangbin Chen, and Michael Lyu. 2025. No More Labelled Examples? An Unsupervised Log Parser with LLMs. Proceedings of the ACM on Software Engineering 2, FSE (June 2025), 2406–2429. doi:10.1145/ 3729377 [18] Hugging Face. 2026. DPO Trainer. https://huggingface.co/docs/trl/dpo_trainer. TRL documentation. Accessed: 2026-10-02. [19] Yintong Huo, Cheryl Lee, Yuxin Su, Shiwen Shan, Jinyang Liu, and Michael R. Lyu. 2023. EvLog: Identifying Anomalous Logs over Software Evolution. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 391–402. doi:10.1109/issre59848.2023.00018 [20] Hyeonbin Hwang, Doyoung Kim, Seungone Kim, Seonghyeon Ye, and Minjoon Seo. 2024. Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewards. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 1444–1466. doi:10.18653/v1/2024.findings-emnlp.78 [21] Zhihan Jiang, Jinyang Liu, Zhuangbin Chen, Yichen Li, Junjie Huang, Yintong Huo, Pinjia He, Jiazhen Gu, and Michael R. Lyu. 2024. LILAC: Log Parsing using LLMs with Adaptive Parsing Cache. Proceedings of the ACM on Software Engineering 1, FSE (July 2024), 137–160. doi:10.1145/3643733 [22] Taeyoon Kim, Woohyeok Park, Hoyeong Yun, and Kyungyong Lee. 2026. Why Do AI Agents Systematically Fail at Cloud Root Cause Analysis? arXiv:2602.09937 [cs.AI] https://arxiv.org/abs/2602.09937 [23] Kimi Team. 2026. Kimi K2: Open Agentic Intelligence. arXiv:2507.20534 [cs.LG] https://arxiv.org/abs/2507.20534 [24] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (Koblenz, Germany) (SOSP ’23). Association for Computing Machinery, New York, NY, USA, 611–626. doi:10.1145/3600006.3613165 [25] Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xiangru Peng, and Jiaya Jia. 2024. Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs. arXiv:2406.18629 [cs.LG] https://arxiv.org/abs/2406.18629 [26] Cheryl Lee, Tianyi Yang, Zhuangbin Chen, Yuxin Su, and Michael R. Lyu. 2023. Eadro: An End-to-End Troubleshooting Framework for Microservices on Multi-Source Data. In Proceedings of the 45th International Conference on Software Engineering (Melbourne, Victoria, Australia) (ICSE ’23). IEEE Press, 1750–1762. doi:10.1109/ICSE48619.2023.00150 [27] Jemin Lee, Sihyeong Park, Jinse Kwon, Jihun Oh, and Yongin Kwon. 2025. Exploring the trade-offs: quantization methods, task difficulty, and model size in large language models from edge to giant. In Proceedings of the ThirtyFourth International Joint Conference on Artificial Intelligence (Montreal, Canada) (IJCAI ’25). Article 902, 9 pages. doi:10.24963/ijcai.2025/902 [28] Zeyan Li, Nengwen Zhao, Mingjie Li, Xianglin Lu, Lixin Wang, Dongdong Chang, Xiaohui Nie, Li Cao, Wenchi Zhang, Kaixin Sui, Yanhua Wang, Xu Du, Guoqiang Duan, and Dan Pei. 2022. Actionable and interpretable fault localization for recurring failures in online service systems. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Singapore, Singapore) (ESEC/FSE 2022). Association for Computing Machinery, New York, NY, USA, 996–1008. doi:10.1145/3540250.3549092 [29] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s Verify Step by Step. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=v8L0pN6EOi [30] Zicheng Lin, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xing Wang, Ruilin Luo, Chufan Shi, Siheng Li, Yujiu Yang, and Zhaopeng Tu. 2025. Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM’s Reasoning Capability. In Forty-second International Conference on Machine Learning. https://openreview.net/forum?id=fnz1g18EdI [31] Dewei Liu, Chuan He, Xin Peng, Fan Lin, Chenxi Zhang, Shengfang Gong, Ziang Li, Jiayu Ou, and Zheshun Wu. 2021. MicroHECL: High-Efficient Root Cause Localization in Large-Scale Microservice Systems. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 338–347. doi:10.1109/ ICSE-SEIP52600.2021.00043 [32] Yu Luo, Jiamin Jiang, Jingfei Feng, Lei Tao, Qingliang Zhang, Xidao Wen, Yongqian Sun, Shenglin Zhang, and Dan Pei. 2026. OpsAgent: An Evolving Multi-agent System for Incident Management in Microservices. arXiv:2510.24145 [cs.AI] https://arxiv.org/abs/2510.24145 [33] Yuan Meng, Shenglin Zhang, Yongqian Sun, Ruru Zhang, Zhilong Hu, Yiyin Zhang, Chenyang Jia, Zhaogang Wang, and Dan Pei. 2020. Localizing Failure Root Causes in a Microservice through Causality Inference. In 2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS). 1–10. doi:10.1109/IWQoS49365.2020.9213058 [34] OpenAI. 2026. OpenAI GPT-5 System Card. arXiv:2601.03267 [cs.CL] https://arxiv.org/abs/2601.03267

22

Kyoohyun Lee, Woohyeok Park, Taeyoon Kim, Sungyong Baik, and Kyungyong Lee

[35] Dan Pei. 2025. AIOps Challenge 2025 Dataset. https://www.aiops.cn/gitlab/aiops-live-benchmark/aiopschallenge2025. CC BY-NC 4.0. Accessed: 2026-08-27. [36] Luan Pham, Huong Ha, and Hongyu Zhang. 2024. BARO: Robust Root Cause Analysis for Microservices via Multivariate Bayesian Online Change Point Detection. Proc. ACM Softw. Eng. 1, FSE, Article 98 (July 2024), 24 pages. doi:10.1145/ 3660805 [37] Luan Pham, Huong Ha, Xiuzhen Zhang, and Hongyu Zhang. 2026. TORAI: Multi-source Root Cause Analysis for Blind Spots in Microservice Service Call Graph. arXiv:2604.13522 [cs.SE] https://arxiv.org/abs/2604.13522 [38] Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In Thirty-seventh Conference on Neural Information Processing Systems. https://openreview.net/forum?id=HPuSIXJaa9 [39] Evelien Riddell, James Riddell, Gengyi Sun, Michał Antkiewicz, and Krzysztof Czarnecki. 2026. Stalled, Biased, and Confused: Uncovering Reasoning Failures in LLMs for Cloud-Based Root Cause Analysis. In Proceedings of the 2026 IEEE/ACM Third International Conference on AI Foundation Models and Software Engineering (FORGE ’26). ACM, 172–183. doi:10.1145/3793655.3793732 [40] Devjeet Roy, Xuchao Zhang, Rashi Bhave, Chetan Bansal, Pedro Las-Casas, Rodrigo Fonseca, and Saravan Rajmohan. 2024. Exploring LLM-Based Agents for Root Cause Analysis. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (Porto de Galinhas, Brazil) (FSE 2024). Association for Computing Machinery, New York, NY, USA, 208–219. doi:10.1145/3663529.3663841 [41] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs.LG] https://arxiv.org/abs/1707.06347 [42] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 [cs.CL] https://arxiv.org/abs/2402.03300 [43] Jin Shieh and Eamonn Keogh. 2008. iSAX: Indexing and mining terabyte sized time series. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Las Vegas, Nevada, USA) (KDD ’08). Association for Computing Machinery, New York, NY, USA, 623–631. doi:10.1145/1401890.1401966 [44] Pan Tang, Shixiang Tang, Huanqi Pu, Zhiqing Miao, and Zhixing Wang. 2025. MicroRCA-Agent: Microservice Root Cause Analysis Method Based on Large Language Model Agents. arXiv:2509.15635 [cs.AI] https://arxiv.org/abs/2509. 15635 [45] Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https://arxiv.org/abs/2505.09388 [46] Upstage. 2026. Solar Pro 4. https://www.upstage.ai/blog/en/solar-pro-4. Accessed: 2026-09-30. [47] Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. MathShepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 9426–9439. doi:10.18653/v1/2024.acllong.510 [48] Ping Wang, Jingmin Xu, Meng Ma, Weilan Lin, Disheng Pan, Yuan Wang, and Pengfei Chen. 2018. CloudRanger: root cause identification for cloud native systems. In Proceedings of the 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (Washington, District of Columbia) (CCGrid ’18). IEEE Press, 492–502. doi:10.1109/ CCGRID.2018.00076 [49] Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022. Finetuned Language Models are Zero-Shot Learners. In International Conference on Learning Representations. https://openreview.net/forum?id=gEZrGCozdqR [50] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS ’22). Curran Associates Inc., Article 1800, 14 pages. [51] Zhe Xie, Zeyan Li, Xiao He, Shenglin Zhang, Longlong Xu, Yuzhuo Yang, Tieying Zhang, Jianjun Chen, Rui Shi, and Dan Pei. 2026. FoundRoot: Towards Foundation Model for Root Cause Analysis via Structured Deep Thinking. In Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering (Rio de Janeiro, Brazil) (ICSE ’26). ACM, 1225–1237. doi:10.1145/3744916.3787814 [52] Junjielong Xu, Qinan Zhang, Zhiqing Zhong, Shilin He, Chaoyun Zhang, Qingwei Lin, Dan Pei, Pinjia He, Dongmei Zhang, and Qi Zhang. 2025. OpenRCA: Can Large Language Models Locate the Root Cause of Software Failures. In The Thirteenth International Conference on Learning Representations. https://openreview.net/forum?id=M4qNIzQYpd [53] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=WE_vluYUL-X

Fault-Free Profiles and Decision-Level Training for Root Cause Analysis via Small Language Models

23

[54] Guangba Yu, Pengfei Chen, Yufeng Li, Hongyang Chen, Xiaoyun Li, and Zibin Zheng. 2023. Nezha: Interpretable Fine-Grained Root Causes Analysis for Microservices on Multi-modal Observability Data. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (San Francisco, CA, USA) (ESEC/FSE 2023). Association for Computing Machinery, New York, NY, USA, 553–565. doi:10.1145/3611643.3616249 [55] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. 2025. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476 [cs.LG] https://arxiv.org/abs/2503.14476 [56] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. STaR: Bootstrapping Reasoning With Reasoning. In Advances in Neural Information Processing Systems, Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.). https://openreview.net/forum?id=_3ELRdg2sgI [57] Chenxi Zhang, Zhen Dong, Xin Peng, Bicheng Zhang, and Miao Chen. 2024. Trace-based Multi-Dimensional Root Cause Localization of Performance Issues in Microservice Systems. In 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE). 1347–1358. doi:10.1145/3597503.3639088 [58] Chenxi Zhang, Xin Peng, Tong Zhou, Chaofeng Sha, Zhenghui Yan, Yiru Chen, and Hong Yang. 2022. TraceCRL: contrastive representation learning for microservice trace analysis. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Singapore, Singapore) (ESEC/FSE 2022). Association for Computing Machinery, New York, NY, USA, 1221–1232. doi:10.1145/3540250.3549146 [59] Lingzhe Zhang, Tong Jia, Yunpeng Zhai, Leyi Pan, Chiming Duan, Minghua He, Pei Xiao, and Ying Li. 2026. HypothesizeThen-Verify: Speculative Root Cause Analysis for Microservices with Pathwise Parallelism. In Proceedings of the IEEE/ACM 48th International Conference on Software Engineering (ICSE-NIER ’26). ACM, 21–25. doi:10.1145/3786582. 3786803 [60] Lingzhe Zhang, Yunpeng Zhai, Tong Jia, Chiming Duan, Siyu Yu, Jinyang Gao, Bolin Ding, Zhonghai Wu, and Ying Li. 2026. ThinkFL: Self-Refining Failure Localization for Microservice Systems via Reinforcement Fine-Tuning. ACM Trans. Softw. Eng. Methodol. 35, 9, Article 295 (Aug. 2026), 44 pages. doi:10.1145/3789262 [61] Xiao Zhang, Qi Wang, Mingyi Li, Yuan Yuan, Mengbai Xiao, Fuzhen Zhuang, and Dongxiao Yu. 2025. TAMO: FineGrained Root Cause Analysis via Tool-Assisted LLM Agent with Multi-Modality Observation Data in Cloud-Native Systems. arXiv:2504.20462 [cs.AI] https://arxiv.org/abs/2504.20462 [62] Nengwen Zhao, Junjie Chen, Xiao Peng, Honglin Wang, Xinya Wu, Yuanzong Zhang, Zikai Chen, Xiangzhong Zheng, Xiaohui Nie, Gang Wang, Yong Wu, Fang Zhou, Wenchi Zhang, Kaixin Sui, and Dan Pei. 2020. Understanding and Handling Alert Storm for Online Service Systems. In 2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). [63] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-bench and Chatbot Arena. In Proceedings of the 37th International Conference on Neural Information Processing Systems (New Orleans, LA, USA) (NIPS ’23). Curran Associates Inc., Red Hook, NY, USA, Article 2020, 29 pages. [64] Tong Zhou, Xin Peng, Jie Zhang, Chaofeng Sha, Chenxi Zhang, Zicheng Yuan, and Senyu Xie. 2026. TraceLLM: Evaluating and Exploring Large Language Models on Trace Analysis in Microservice-based Web Applications. In Proceedings of the ACM Web Conference 2026. ACM, 5120–5131. doi:10.1145/3774904.3792164 [65] Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Chao Ji, Wenhai Li, and Dan Ding. 2021. Fault Analysis and Debugging of Microservice Systems: Industrial Survey, Benchmark System, and Empirical Study. IEEE Transactions on Software Engineering 47, 2 (Feb. 2021), 243–260. doi:10.1109/tse.2018.2887384 [66] Xiang Zhou, Xin Peng, Tao Xie, Jun Sun, Chao Ji, Dewei Liu, Qilin Xiang, and Chuan He. 2019. Latent error prediction and fault localization for microservice applications by learning from system trace logs. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’19). ACM, 683–694. doi:10.1145/3338906.3338961

Related documents

• Free to Look, Free to Reason: Simple LLM Harnesses Outdiagnose Hand-Crafted Root Cause Analysis Agents#1459290
• Free to Look, Free to Reason: Simple LLM Harnesses Outdiagnose Hand-Crafted Root Cause Analysis Agents#1450732
• Beyond Fault Localization: A Trajectory-Level Study of LLM Agents for Microservice Root Cause Analysis#1298102
• Beyond Fault Localization: A Trajectory-Level Study of LLM Agents for Microservice Root Cause Analysis#480119
• Beyond Conditional Independence: Root Cause Analysis with Deep Causal Models#1131972