Agar: What-If Analysis for Microservice Applications

AGAR: What-If Analysis for Microservice Applications

arXiv:2610.36636v1 [cs.DC] 27 Oct 2026

Saarth Deshpande, Neeraja J. Yadwadkar∗, Venkat Arun∗ The University of Texas at Austin

Abstract

often hard to know without deploying it. This is especially expensive in microservices, where local changes rarely stay local: changing one component can shift bottlenecks across the application. Performance can also change with the underlying environment, with different hardware surfacing failures that did not appear elsewhere. Because the impact of a choice must ultimately be established by observing application behavior over time [29, 54], evaluating each choice requires deploying and exercising the application under representative conditions, making exploration across many configurations expensive [52]. Operators need to know whether a proposed change will actually solve an observed performance problem, and what it will cost, without having to deploy the change. This requires reasoning about how the application would behave under alternatives that have not been tried out yet. Would adding more resources restore SLO compliance or is the bottleneck elsewhere? Is the SLO unattainable under the current workload regardless of resource allocation? How much cost could be saved if we relaxed the SLO by a certain amount? Would optimizing one component reduce the overall resource needs of the application? These are counterfactual questions: they ask what would have happened under a change that was not actually made. A growing body of systems work has explored counterfactual prediction for networks [2,11], storage [45,61], and application behavior [18, 50, 60]. These systems show that learned models can predict behavior under decisions that were not actually exercised, but applying this idea to microservices exposes a fundamental scaling challenge. An end-to-end model must directly learn application behavior as a function of the workload, resource allocation, and an SLO target to effectively capture the joint behavior of many interacting microservices over a combinatorial space of conditions, many of which may never have been exercised. A natural alternative is to decompose the application into per-component models. This narrows the behavior that each model must capture, but creates a different challenge: prediction errors compound as models are composed. With 50

Microservice operators must reason about how applications would behave under changes that have not yet occurred: different workloads, resource allocations, or SLO targets. Evaluating every alternative experimentally is expensive, but predicting such counterfactuals is difficult because microservices contain many interacting components. End-to-end models must learn behavior over a combinatorial space of conditions, but composing per-component predictors causes prediction errors to accumulate. We address this challenge by changing what each component model must predict. Rather than predicting exact values, we learn simple, interpretable bounds that contain every observed behavior while remaining tight enough to support useful end-to-end conclusions. We build AGAR, a counterfactual performance analysis tool that composes these models to answer end-to-end SLO-feasibility questions. This allows us to formulate autoscaling as a counterfactual query: what is the smallest resource allocation under which an SLO is met? Our evaluation demonstrates that across three microservice applications, AGAR’s bounds are actionable: its derived autoscaler reduces CPU allocation by up to 36.8% over stateof-the-art horizontal autoscalers while satisfying application SLOs. Further, AGAR is able to predict operational cost given workload traces, provide guidance on which service’s startup time to optimize and localize performance bugs within each application.

1

Introduction

Microservice operators constantly make decisions that can alter application performance: how many resources to allocate, how services are configured, what hardware to run them on [65], and which service’s performance to optimize [60]. A wrong choice can waste resources [24, 57], violate Service Level Objectives (SLOs) [10, 47, 50], or fail to address the actual bottleneck [18, 58]. However, the effect of a choice is ∗ Equal advisory contribution.

1

CPU utilization (%)

component models, for example, naively guaranteeing 95% confidence that the composition is correct would require each model to be correct roughly 99.9% of the time.

100

240

480 720 960 Time (s) (a) Backpressure keeps frontend CPU above the HPA threshold. 30 25 20 15 10 5 0

HPA AGAR

frontend slow-service

240

480 720 960 Time (s) (b) HPA scales frontend while AGAR scales the bottlenecked slow-service. HPA AGAR

6 4 2 0

We jointly fit the component models to minimize uncertainty in the end-to-end quantity predicted by their composition—for example, the average resource cost of meeting a given SLO—while requiring each model to fit all observed data. We prove that this optimization problem is NP-hard (Appendix B). We develop an efficient greedy algorithm with a log(n) + 1 approximation guarantee for n data points, and show that it performs well in practice.

50% threshold

50

p95 latency (s)

Making uncertainty explicit also allows us to use simple models motivated by system intuition. For example, we might model average CPU utilization as inversely proportional to the number of CPUs allocated. Such a model does not explicitly capture effects such as cross-VM interference or sudden bursts of garbage collection; instead, their observed effects are absorbed into the model’s uncertainty bounds. More expressive models, including large neural models or LLM-based predictors, could attempt to capture these effects directly, but doing so introduces additional data requirements and complexity that are unnecessary for our goal. We do not need to predict behavior exactly; we need bounds that are tight enough to support useful counterfactual conclusions. Simple models can provide such bounds with less data, while remaining easier to interpret and compose across services.

slow-service frontend

150

Replicas

The key is to change what we require from each component model. Rather than requiring a model to predict a single value, such as CPU utilization, perfectly, we make its uncertainty explicit. For example, the model may predict that utilization lies between, say, 25–35%. This allows models to achieve 100% accuracy. Unfortunately, they may do so vacuously by always predicting a range of 0 to ∞. In practice, however, our models achieve 100% accuracy while remaining precise enough to be useful. This not only addresses the model-composability problem, but also makes model errors and data drift easy to detect: the first observation that falls outside a predicted range signals that the model must be retrained.

200

240

480 720 960 Time (s) (c) AGAR keeps p95 latency below 500 ms by scaling slow-service.

Figure 1: Backpressure case study. (a) Under constant load, frontend workers busy-spin on the CPU while awaiting responses from slow-service, causing CPU utilization to remain above its HPA threshold longer. (b) HPA consequently scales frontend to ∼18 replicas while adding only one slow-service replica, leaving the backend bottleneck unresolved. (c) p95 latency climbs to ∼6 s. AGAR instead identifies slow-service as the bottleneck and scales it out, keeping p95 below 500 ms after its models converge.

Guided by these insights, we introduce AGAR, a counterfactual performance analysis tool to answer end-to-end SLO-feasibility questions. AGAR learns per-component models that fit every observation, then composes them to reason about hypothetical changes in workload, resource allocation, and SLO targets. Using these models to identify the smallest resource allocation that can satisfy a target SLO directly yields an autoscaler. Across three microservice applications, we demonstrate that the bounds found by AGAR are actionable. Using these bounds, AGAR’s autoscaler reduces CPU allocation by up to 36.8% over state-of-the-art horizontal autoscalers while satisfying application SLOs. When the bounds fail to hold, they expose previously unknown performance problems in all three applications (see §5.3).

2

Motivation

Running a microservice application requires making many performance-critical choices: how many resources to allocate to each service, how to configure scaling policies, what Service Level Objective (SLO) to target, and whether changes to the application or deployment are worth making. Answering these questions today often requires trying the corresponding configuration or change and measuring what happens [52]. For applications with many interacting services, repeatedly doing so can require substantial experimentation. 2

Consider the amount of tuning required to even configure an application for one workload and SLO. The Kubernetes Horizontal Pod Autoscaler (HPA) [35], the most widely used autoscaler today [16], requires operators to specify per-service scaling policies, such as what CPU or memory utilization should trigger scaling and how long that utilization must persist. For example, “scale up if CPU utilization exceeds 60% for more than 30 s or memory utilization exceeds 80% for 10 s” for each service. Microservice applications may contain tens, hundreds, or even thousands of services [9]. Moreover, a configuration tuned for one set of conditions may perform poorly as those conditions change. This is not a one-time cost: Autopilot [48] reports Google services whose changing resource demands required operators to manually retune CPU and memory allocations 8–12 times per month. This tuning is further complicated by interactions between services. For example, when a backend service becomes a bottleneck, its frontend service must wait for responses, creating backpressure—delays that propagate from downstream to upstream services. The resulting backpressure can affect the frontend’s resource utilization even though scaling the backend is what relieves the bottleneck (Figure 1). Such interactions make it difficult to determine which service should be scaled from utilization alone [19]. Learning-based resource managers [39, 47, 49, 50, 57, 63] can automate part of this tuning, using application measurements to guide per-service resource allocation through horizontal or vertical scaling and reduce resource use while targeting end-to-end latency SLOs. But operators often need answers to questions that go beyond finding a configuration for the application as it runs today. Suppose an engineer could reduce the startup time of one service. Would that change meaningfully reduce resource cost or improve SLO satisfaction? Suppose the traffic mix shifts toward one endpoint1 . Will the current configuration still suffice, and if not, how should resource allocation change? Suppose the SLO is tightened or relaxed. What would that do to the resources required to meet the new SLO? Evaluating every such possibility by deploying it and re-tuning the application turns each question into another experiment. Consider what happens when the workload changes. A shift in traffic mix changes how requests flow through the application’s services and can move bottlenecks from one component to another. Before we can answer any counterfactual question, the system must be re-tuned to the new workload. HPA [35] leaves this entirely up to the developer. Erlang [49] helps, but requires a new instance of the application that is not serving user traffic to experiment on and train whenever the workload moves outside its training parameters. Erlang therefore still relies on active experimentation to learn how the application behaves under new conditions. In contrast, AGAR can learn passively from observed data

and reuse what it has learned across conditions. The key is that AGAR models simple, per-service relationships and composes them to answer end-to-end questions. For example, a change in traffic mix may alter how much load reaches a service without changing how its utilization responds to load and resource allocation. AGAR can therefore reuse the models that map each service’s load to its utilization to determine SLO satisfaction under the new workload rather than retraining the application from scratch. When such reuse is no longer valid, AGAR’s “always accurate” models make this immediately detectable: the first observation that violates a learned relation identifies a model that must be retrained. Such violations can occur when the deployment environment changes. For example, when we moved Online Boutique [23] and TeaStore [56] from AWS to CloudLab [17], CloudLab’s slower storage exposed performance problems that had not appeared on AWS, invalidating relationships learned from the earlier deployment (see §5.3). In our experiments, such retraining was needed only when an environment change exposed previously unseen performance bugs. This allows AGAR to make predictions far from prior experiments without requiring a new tuning run for every workload, SLO target, or candidate change. Other systems reduce the amount of experimentation in a different way: they restrict the dimensionality or locality of the problem being learned. Galileo [50], a vertical autoscaler2 , deliberately models only the effect of local changes to resource allocations. This allows it to make linearizing assumptions and use compressed sensing to reduce the number of experiments needed to identify the bottleneck. AutoThrottle [57] similarly reduces the search space by dividing services into two groups and tuning one throttling target per group to guide per-service CPU allocation, while Slowpoke [60] answers counterfactual questions about optimizing a given service by actively slowing down every other service. These systems still rely on active experiments, but keep the learning problem tractable by narrowing the space over which they must reason. These observations suggest a different point in the design space: a system should reuse simple component-level relationships across workloads and configurations, compose them to answer end-to-end counterfactual questions, and detect when those relationships no longer hold rather than requiring active experimentation whenever conditions change. AGAR is designed around these principles. We next describe how AGAR represents component behavior, learns these models from passive observations, and composes them to answer counterfactual queries.

1 An endpoint is a distinct externally visible API or RPC call of an appli-

2 Vertical autoscaling manages resources allocated to a single VM. Hori-

3

AGAR

AGAR is a counterfactual performance analysis tool for microservice applications that answers SLO-feasibility ques-

cation, each with its own latency SLO.

zontal autoscaling, our focus, allocates the number of VMs.

3

AGAR Engine

Training Data

Application Behavior

SLO model

Utilization model

AGAR’s Autoscaler

Inconclusive? Why?

Violation → retrain

Developer

Can an SLO of 300 ms be achieved for this workload?

U, RPS, τ

scales

Application

New params

RPS model

If yes, at what cost?

params

Workload

observes U’, RPS’

AGAR’s Answer

AGAR Engine see Figure 2

Example Counterfactual Query

Figure 3: AGAR autoscaler control loop. The developer provides the initial observed utilization U, RPS, latencies and SLO target τ to fit the AGAR Engine’s models. At runtime, the autoscaler periodically observes U’ and RPS’ from the cluster, and uses the learned parameters (α(·), β, ε, τ, ∆T (·)) to answer the counterfactual query: given U ′ and RPS′ , what is the minimum replica count that satisfies τ? When a model bound is exceeded or an SLO violation occurs, it retrains by calling the AGAR Engine to update its parameters.

Figure 2: AGAR system overview.

tions under hypothetical changes to workload, resource allocation, and SLO targets. To answer these questions, AGAR fits models that relate workload and resource allocation to each microservice’s resource utilization, and that utilization to end-to-end SLO satisfaction.

3.1

AGAR Architecture

Recall that these are relations, not exact predictions. For example, rather than predicting that a particular workload will result in exactly 60% CPU utilization, we might say that its CPU utilization should fall between 50% and 70%. This distinction lets us learn relations that hold for 100% of the training observations.

Figure 2 shows the architecture of AGAR. Users interact with AGAR through a fixed set of counterfactual queries: how many resources are needed to meet a given SLO; how that resource requirement changes for a different workload or SLO; and how changing a service property, such as its startup time, affects the resources needed to meet the SLO. To answer these questions, AGAR uses per-endpoint request rates and latencies and per-service CPU and memory utilization collected from previous executions. AGAR uses these measurements to fit the models described in the following sections. Together, these models comprise the AGAR Engine. Autoscaling is one such query issued repeatedly: given predicted request rates and a target SLO, AGAR computes how many replicas each service needs to meet the SLO. As the workload changes, AGAR repeats this query to determine the next resource allocation. During autoscaling, runtime monitors check new request-rate, utilization, and latency observations against the learned relationships and trigger retraining when an observation falls outside the corresponding model bounds, as shown in Figure 3.

3.2

Utilization Model. To retain this ability to fit all observed behavior, we deliberately make weak assumptions about application behavior. To model the utilization Ui of service i, we assume that it comprises three components: a workloaddependent component, ∑ j αi, j , divided evenly across Pi replicas, where αi, j is the utilization contributed by endpoint j to service i and increases monotonically with its request rate (RPS), a workload-independent component, βi ≥ 0, and bounded variation εi ≥ 0. Specifically, we assume the following relation: Ui ∈

∑ j αi, j (rps j ) + βi ± εi Pi

(1)

Importantly, we make no assumption about the shape of αi, j beyond monotonicity: different endpoints and services can therefore exhibit different relationships between request rate and utilization. We use this relation for both CPU and memory utilization. Different services exercise this flexibility in different ways. Some services can retain memory in heaps and caches even when request rate is low or zero, resulting in a substantial workload-independent component captured by βi . Garbage collection, for example, can introduce large, irregular changes in CPU and memory utilization, which manifest as a larger εi . Thus, the same relation can capture qualitatively differ-

Models and Modeling Philosophy

We describe application behavior under different workloads and resource allocations using three models: (1) the Utilization Model that relates per-endpoint request rates and each service’s replica count to its CPU and memory utilization; (2) the SLO Model that relates these utilizations to whether each endpoint’s SLO is met; and (3) the RPS Model that relates past request rates to future demand. Figure 2 shows how these models compose within the AGAR Engine. 4

Service B utilization (%)

RPS Model. Finally, to account for service startup delays (∆C j ), AGAR predicts future request rates so that replicas can be provisioned before the demand arrives. Like HPA, which reacts to the maximum utilization over its behavior.scaleUp.stabilizationWindowSeconds [34] window, we use the maximum request rate over the previous t seconds. The simplicity is intentional: AGAR requires a prediction of future demand, but not a particular prediction method. More sophisticated predictors can therefore be substituted directly. Despite its simplicity, future request rate exceeds our prediction in at most E1 % observations, where E is the number of application endpoints.

The models above work together to answer AGAR’s counterfactual queries. Consider determining how many replicas each service needs to meet an SLO τ. AGAR first predicts the request rate at the end of each service’s startup delay. Given these predicted request rates, the utilization model bounds each service’s CPU and memory utilization for a candidate replica count. The SLO model then determines whether these utilization bounds are low enough to satisfy τ. Let Ti denote an SLO-safe utilization threshold for service i, and let rpsmax denote the predicted request rate for endpoint j j. For the SLO to be satisfied, the upper bound on service i’s utilization must remain below Ti . Applying this condition to Equation 1 gives the required replica count: ∑ j αi, j (rpsmax j ) Ti − βi − εi

60 20 0 0 20 99 60 Service A utilization (%) (1) PA = 2, PB = 1 99 60 20 0 0 20 99 60 Service A utilization (%) (3)

PA = PB = 1 99 60 20 0 0 20 99 60 Service A utilization (%) (2) PA = 2, PB = 3 99 60 20 0 0 20 99 60 Service A utilization (%) (4)

Figure 4: Illustration of selecting utilization bounds for SLO satisfaction. Crosses denote SLO violations observed at corresponding service utilizations, the shaded region denotes utilizations predicted to satisfy the SLO, and PA and PB denote the replica counts for services A and B under the corresponding utilization bounds. We (1) optimistically start with 99% utilization bounds, (2) discard violations dominated by another violation, since excluding the latter also excludes them, and (3) tighten utilization bounds at the lowest additional replica cost per newly excluded SLO violation. (4) We repeat until all observed violations lie outside the shaded region.

3.3 Composing and jointly optimizing the models

Pi =

99

Service B utilization (%)

SLO Model. We next apply the same philosophy to SLO satisfaction: instead of predicting latency, we identify utilization bounds within which an SLO is satisfied. Our core assumption is that there exists a utilization below which a target SLO is always satisfied. Above these bounds, observations may either satisfy or violate the SLO, and we make no claim about them. Figure 4 illustrates the intuition behind constructing these bounds. Any bounds that exclude all observed SLO violations optimistically treat the enclosed region as SLO-satisfying, even if no observations fall inside it. There can be many such bounds, and we describe how AGAR selects among them in §3.3.

PA = PB = 1

Service B utilization (%)

Service B utilization (%)

ent utilization behavior across services without changing its structure.

sequence of linear programs to find the optimal solution in polynomial time (Appendix A). The harder problem is choosing the SLO-safe utilization thresholds. Many choices can exclude all observed SLO violations, but they can require different numbers of replicas. For example, suppose an SLO violation occurs when one service has high CPU utilization and another has high memory utilization. Lowering either service’s corresponding threshold can exclude that violation, but each choice can imply a different replica cost through Equation 2. Thus, there can be multiple ways to exclude each violation, with different costs. Choosing the optimal set of thresholds is NP-hard (proof in Appendix B). However, we develop a simple greedy algorithm that we prove finds a solution within a factor of log(n) + 1 of the optimal number of replicas needed, where n is the number of SLO violations in training data. AGAR’s greedy algorithm is inspired by the standard greedy set-cover approximation [14]. AGAR initially sets each uti-

(2)

Once the SLO model produces Ti , AGAR fits the utilization model to minimize the replica count in Equation 2. The resulting utilization-model optimization is fractional, so we use the Charnes–Cooper transformation [12] to convert it into a 5

RPS

ments in §5.2, full retraining was required at most once while utilization bounds were updated throughout each run.

AGAR’s Autoscaler kubectl top

Kubernetes

Pt

HPA Controller

Utilizations

Utilization-bound violation. After each tick, AGAR’s Autoscaler checks whether the observed utilization Ut lies within the bounds predicted by Equation 1. A violation indicates that the learned parameters no longer describe the service’s behavior — for example, because the workload has shifted or pods have migrated to different nodes. AGAR’s Autoscaler responds by retraining the utilization model in the background, updating its parameters, and reloading them into the live control loop. A per-metric cooldown of 60 seconds prevents retrain storms when violations are transient.

Metrics Server

Deployment

Figure 5: AGAR’s Autoscaler sits outside the Kubernetes control plane (dashed region). At each tick, it reads per-service utilization via kubectl top from the Metrics Server and patches the service’s HPA object to Pt , causing HPA to adjust the service’s Deployment to Pt pods.

SLO violation. Every 60 seconds, a separate monitor checks whether any end-to-end latency measurement exceeds the SLO target τ. An SLO violation for any endpoint leads to a full-application retrain: AGAR’s Autoscaler calls the AGAR Engine (Figure 3) to re-fit all service models from the accumulated data and pushes updated thresholds to the deployed application. Full retrains are rate-limited to one per five minutes; if a retrain is already in progress, the request is dropped and the next monitoring cycle re-evaluates. In both cases, the retrain writes updated parameters to disk; per-service utilization monitors detect the change and reload before the next tick.

lization bound to its largest valid candidate, up to 99%. Each service’s initial replica budget is the maximum replica requirement across its utilization dimensions. At each step, AGAR updates one service’s replica budget, choosing the move with the smallest additional replica cost per newly excluded violation. It fits utilization models at candidate thresholds on demand to determine this move. Once all violations are covered, AGAR selects the final thresholds for each updated service. For each utilization dimension, it selects the threshold that excludes the most retained SLO violations within the service’s budget, choosing the largest valid threshold when several exclude the same violations. Importantly, this optimization does not change the assumptions made by the individual models. It selects, among models that fit all observed behavior, those whose composition gives the most useful answer to the end-to-end query.

4

AGAR’s implementation separates offline model fitting from online autoscaling. We implement model fitting in Python using PuLP [46] with the CBC solver. For online operation, each service runs its own autoscale loop in a separate process. Utilization-bound retrains for a single service are submitted asynchronously to a single-worker executor, so retrains are serialized but never block the tick loop. Full retrains are guarded by a cross-process lock so that a slow full retrain neither races with a concurrent single-service retrain nor starves the autoscale loops waiting on updated parameters. A shared tick-coordinator thread snapshots RPS and latency readings atomically at each 15-second boundary, ensuring all per-service loops operate on a consistent view of the cluster state.

Implementation

AGAR’s Autoscaler runs as a per-service control loop on a 15second tick, matching HPA’s default scrape granularity. Like HPA, it operates per-service and reads cluster metrics via the Kubernetes metrics API. AGAR’s Autoscaler piggybacks on the HPA (Figure 5): at each tick, it evaluates Equation 2 to obtain the minimum replica count Pt for service s and patches the existing HPA object by pinning both minReplicas and maxReplicas to exactly Pt , while leaving all underlying Kubernetes pod management intact. To compute Pt , AGAR’s Autoscaler evaluates Equation 2 using the current RPS observed from the cluster and the parameters learned offline by the AGAR Engine. If no feasible model exists for a service, AGAR’s Autoscaler falls back to the HPA. However, we have not encountered this case in practice. Because the parameters are fit offline on historical data, AGAR’s Autoscaler monitors the application’s live observability data for conditions that indicate the model no longer reflects the application’s current behavior and triggers a retrain when they arise, off the critical path. Across all experi-

As AGAR’s Autoscaler operates, it continuously appends observations to the training set. As the training set grows to cover more operating conditions, the model fits tighten and bound violations become increasingly rare, and the autoscaler settles into a steady state where Pt changes only in response to genuine shifts in demand.

5

Evaluation

Our evaluation demonstrates AGAR’s use across autoscaling, performance diagnosis, and counterfactual changes to workloads, resource allocation, and SLO targets. 6

5.1

Experimental Setup

nal. These HTTP entrypoints provide the most direct measure of offered load. Using call-rate measurements from internal service interactions would instead make the workload signal depend on application-specific implementation details rather than externally imposed demand. Since the original requestrate data is available at 60 s granularity, we linearly interpolate it to 15 s resolution for replay. We assign one such workload trace to each application endpoint and rescale the resulting demand to match the capacity of the target deployment. Locust is configured with 100 users and a ramp-up of 1 user/s for TeaStore and Online Boutique, and with 1000 users and a ramp-up of 100 users/s for Social Network.

Cluster. We run experiments on two otherwise identical AWS EC2 clusters, each comprising 15 t3.2xlarge instances3 : one control-plane instance and 14 worker instances running Kubernetes deployed with kubeadm. Application pods are scheduled only on the worker instances; AGAR’s Autoscaler runs on the control-plane instance. One cluster runs on an EC2 Dedicated Host [3], avoiding host-level interference from colocated instances, while the other uses shared tenancy and is exposed to such interference. The dedicated and shared clusters also let us evaluate AGAR under hostlevel interference. AGAR does not model noisy neighbors explicitly; instead, their effects appear in the utilization relationships learned from each environment. Short-term interference broadens the range of utilization observed for a given workload, while sustained interference shifts the learned baseline or workload-utilization relationship. We quantify these parameter changes in Appendix E.

Baselines. We compare AGAR against Erlang [49], a banditlearning horizontal autoscaler, and against the Kubernetes Horizontal Pod Autoscaler (HPA) [35] configured with handtuned utilization thresholds and scaling policies. Most prior research focuses on vertical autoscaling [50,57,63]. We therefore use Erlang as our research baseline for horizontal autoscaling. Our HPA configurations are the best we are aware of after extensive experimentation, obtained by iteratively adjusting thresholds and observing scaling behavior under load. Many production horizontal autoscalers, like HPA, leave users to tune scaling targets for their applications [6, 21, 22, 33].

Benchmarks. We evaluate on three microservice applications: TeaStore [56] (7 services), Online Boutique [23] (11 services), and Social Network [19] (20 services). For Social Network, we exclude MongoDB instances from the autoscaled service set. Although these components are deployed as Kubernetes Deployment objects rather than StatefulSet4 objects, the deployment does not support sharding, so multiple replicas would contend for the same persistent volume. We also exclude the Jaeger distributed tracing service. For Online Boutique, we replace the bundled load generator with an external Locust [37] instance. Kubernetes configurations contain a guaranteed CPU and memory allocation (requests), and a maximum allocation that may be reached if resources are available (limits). For Online Boutique, each replica is configured with CPU requests and limits of 1000m and 2000m respectively, and memory requests and limits of 64 MiB and 128 MiB. The exception is redis-cart, which retains its default configuration: CPU requests and limits of 70m and 125m, and memory requests and limits of 200 MiB and 256 MiB. TeaStore uses the same CPU configuration, but sets both memory requests and limits to 2 GiB. Social Network uses CPU requests and limits of 500m and 1000m respectively, and memory requests and limits of 128 MiB and 256 MiB.

Training and testing. For each application, AGAR’s models are fit on the first 6 of 12 hours of the Alibaba-derived workload [38], and Erlang is trained on the RPS contexts of the same duration. Autoscaler performance is evaluated on the following 2-hour period of each trace; representative workload segments are shown in Appendix C. While the models converge within 4 minutes for Online Boutique and Social Network and 12 minutes for TeaStore, we report results for the entire duration, including the period before convergence. Across all experiments, full retraining was required at most once while utilization bounds were updated throughout each run.

5.2

AGAR as an Autoscaler

We set a 300 ms p95 latency SLO and consider a violation when any endpoint’s p95 latency exceeds this target. Erlang instead uses a single p95 latency computed over requests across all endpoints as its latency signal, so an endpoint can violate the 300 ms SLO without necessarily causing Erlang’s measured p95 to exceed 300 ms, as we show below. HPA is not SLO-aware and reacts directly to resource-utilization targets; consequently, satisfying its utilization targets does not imply SLO compliance.

Workloads. For each application, we derive replay workloads from Alibaba production traces [38], and use the HTTPentrypoint microservice call rate (MCR) as the workload sig3 Standard t3 instances allow noisy-neighbor CPU steal to manifest,

unlike fixed-performance instances whose vCPUs are pinned to physical cores [4]. These instances also experience throttling when their CPU credits are exhausted [5]. 4 A StatefulSet assigns each pod a stable network identity and persistent storage, making it the appropriate abstraction for stateful workloads; a Deployment treats all pods as interchangeable.

Online Boutique. AGAR keeps per-endpoint p95 latencies below 50 ms on dedicated hosts and 100 ms on shared hosts, well below the 300 ms SLO. In contrast, Erlang can miss 7

AGAR

Erlang

Request rate (RPS)

Median p95 latency (ms)

E1 RPS Total RPS

Shared Host

HPA

1.2K 900 600 300 0

10

20

30

Median p95 latency (ms)

2,100

600

1,500

400

900

200 300 0 53 54 Time in trace (min)

55

40

Replicas (a) Online Boutique

Figure 7: Workload transition for Social Network: E2’s request rate rises as E3’s and the aggregate request rate decline. Since Erlang scales based on aggregate request rate, E2’s p95 latency exceeds the 300 ms SLO.

1.2K 900 600

endpoints constitute more than 1 − p of traffic—where p is the tail latency percentile—will expose this limitation again. Figure 6 reports the endpoint-level latency and corresponding average replica cost for each autoscaler.

300 0

5

10

20

30

40

Replicas (b) Social Network Median p95 latency (ms)

E3 RPS E2 p95 latency

800

52 5

E2 RPS

p95 latency (ms)

Dedicated Host

TeaStore. With Erlang, the same problem manifests in TeaStore, though in a more extreme form. Two endpoints account for <0.9% and <0.4% of total traffic. Requests for these endpoints time out, violating the 300 ms SLO. However, at under 1.3% of total traffic, their requests fall above the 95th percentile of the aggregate latency distribution—well out of reach of Erlang’s p95 reward signal. The aggregate p95 is determined entirely by four other endpoints that collectively account for over 98% of traffic—and registers no violation. Erlang therefore has no incentive to provision for the violating endpoints and converges to a policy that satisfies the aggregate target while leaving them in violation. The difference is one of degree: in Online Boutique, latency violations at infrequently requested endpoints are masked by the larger volume of lower-latency requests, whereas in TeaStore the violating endpoints are so rare that they lie entirely beyond the percentile Erlang optimizes for.

60K 900 600 300 0

5

10

20

30

40

Replicas (c) TeaStore

Figure 6: Per-endpoint median p95 latency when using HPA, Erlang, and AGAR on AWS EC2 t3.2xlarge instances with Dedicated and Shared Host tenancy. The horizontal dashed line marks the 300 ms SLO. For each application, the boundary boxes mark the average replica count for each autoscaler–environment pair.

Social Network. Around 53 minutes into the run, endpoint E2’s request rate rises from roughly 40 to 250 RPS, while E3’s falls from roughly 400 to 150 RPS and the aggregate request rate declines, as shown in Figure 7. During this period, AGAR adjusts its replica allocation as demand shifts between endpoints. Erlang’s allocation remains unchanged due to reduction in total RPS: its scaling trigger uses aggregate RPS and ignores endpoint mix and latency. E2’s recorded cumulative p95 rises above the 300 ms SLO and stays above it until the end of the run, yielding median values of 1.1 s on dedicated hosts and 1.2 s on shared hosts.

severe latency violations at infrequently requested endpoints. On shared hosts, for example, an endpoint that constitutes less than 0.6% of requests reaches a median p95 latency of ∼1.1 s and violates the SLO, even though Erlang’s overall p95 latency remains below it. Because Erlang computes this percentile across all endpoints rather than separately for each endpoint, sufficiently rare slow endpoints can be masked by the larger volume of lower-latency requests. Erlang cannot distinguish between endpoints, and any workload where fast 8

5.3

AGAR as a Diagnostic Tool

when the HPA was configured with minReplicas: 10. Yet overprovisioned resources did not solve the underlying problem. Increasing the probe timeout to match the service’s actual startup latency immediately stabilized the pool, restored normal load distribution, and satisfied the SLO with the minimum replica count prescribed by AGAR. This example shows that autoscaling was not the solution. The violation was not fundamentally a capacity problem, but an environment-dependent configuration problem: probe timeouts that were sufficient on AWS became too short on CloudLab, where slower storage increased service startup times. AGAR helped localize the root cause and avoid unnecessary scaling.

While AWS serves as our primary environment for evaluating autoscaling, the applications exhibited diverse performance problems on a CloudLab [17] cluster of r320 machines. The Online Boutique (§5.3.1) and TeaStore (§5.3.2) issues did not occur on AWS, which we attribute to its substantially faster storage; the Social Network issue (§5.3.3) occurred in both environments. Thus, the case studies span issues that arise from interactions with the underlying infrastructure as well as those that persist across environments that AGAR was able to diagnose. 5.3.1

Online Boutique 5.3.2

We begin this case study with a baseline comparison: we first observe how standard autoscalers behave under the workload, then use AGAR to perform counterfactual diagnosis and isolate the structural cause. When we ran Alibaba workloads for Online Boutique, SLO violations were observed for two endpoints. Both HPA and Erlang responded to these violations by scaling out: HPA increased replica counts for services exceeding their CPU thresholds, while Erlang adjusted replica counts across services according to its learned autoscaling policy. Despite this, both approaches failed to alleviate SLO violations: as load increased, newly scheduled replicas could not serve traffic until their initialization completed, concentrating load on already-running replicas and causing queue buildup. AGAR took a different approach. Since no observed configuration satisfied the SLO, we reasoned counterfactually by fitting the utilization models assuming Ti = 100.0 for every service i (see §3.3) — the most optimistic possible scaling threshold to meet an SLO. Even under this assumption, recommendationservice, payment service, and currencyservice exhibited anomalously high β values – high CPU utilization kept triggering scaleouts, but newly scheduled replicas that failed their probes served no requests. This directed our attention to these services’ configurations rather than their replica counts. The root cause was a probe timeout misconfiguration. Although the same configuration worked on AWS, CloudLab’s substantially slower storage—roughly 153 IOPS compared with 1636 IOPS on AWS—increased service initialization times enough for the probes to time out before replicas became ready. Kubernetes readiness probes [15] with insufficient timeouts prevented replicas from receiving traffic, concentrating load on the replicas that do pass the probes. Since load was concentrated on fewer replicas than were provisioned, the model underestimated how much CPU utilization was due to request load, and the remainder appeared as inflated β. Liveness probe failures compounded this by triggering CrashLoopBackOff restarts, further destabilizing the pool. Since each new replica was susceptible to probe failure, the SLO was not violated

TeaStore

When we ran Alibaba workloads for TeaStore, SLO violations were observed across all endpoints. HPA continued scaling out teastore-webui, TeaStore’s client-facing frontend, in response to its high CPU utilization. The SLO was eventually satisfied after a prolonged queue buildup period and at significant replica cost. From HPA’s perspective, the system was simply underprovisioned. Since the SLO was eventually satisfied after the queue buildup released, our SLO model produced SLO-safe scaling thresholds for all TeaStore services. When we fit the utilization models (as described in §3.3) with these thresholds and the corresponding observed workload, teastore-webui emerged as the sole outlier — all other services exhibited normal β and ε values, while teastore-webui showed anomalously high ε on both CPU and memory utilization. The root cause was an unbounded JVM heap. Without explicit heap limits, the JVM expanded the heap and metaspace to the container’s memory limit, causing garbage collection (GC) cycles to grow unpredictably and consume substantial CPU independent of request load. This configuration did not violate the SLO on AWS, but CloudLab’s substantially slower storage amplified the resulting GC overhead and prolonged queue buildup, exposing the problem. The resulting utilization variability manifested as anomalously high ε in the utilization model. Unlike the probe misconfiguration—where load concentration on fewer replicas artificially inflated β—here the erratic GC cycles directly drove utilization variability. Bounding the heap to three-quarters of the container’s memory limit and metaspace to one-eighth via JAVA_TOOL_OPTIONS [41] ensured predictable GC behavior, making CPU utilization a function of application load rather than JVM ergonomics. 5.3.3

Social Network

When we ran Alibaba workloads for Social Network with HPA, the SLO was satisfied throughout — from the perspective of any existing tool, the system was healthy. Yet when AGAR fit its models on the collected data, user-timeline-redis exhibited anomalously high ε on 9

memory utilization. Unlike the Online Boutique and TeaStore issues, this anomaly appeared on both CloudLab and AWS, showing that it was not confined to the slower CloudLab environment. The anomaly is explained by the memory utilization pattern visible in Figure 8: memory climbs steadily, punctuated by sharp drops each time HPA scales out and dilutes the per-pod average, before resuming its climb.

Simulated

Real

35

Cost (vCPUs/hr)

30 25 20 15 10 5 0

Dedicated

Shared

Figure 9: Cost to meet a 200 ms p95 latency target for Online Boutique across dedicated and shared environments. Cost is based on the hourly number of vCPUs required to support AGAR’s prescribed replica configuration. Real costs are higher due to online retraining. Figure 8: Memory utilization and replica count for user-timeline-redis. Memory grows monotonically with data, triggering repeated HPA scale-outs that provide only temporary relief before memory resumes climbing.

workload at these SLO targets. Figure 9 compares predicted and measured costs to meet a 200 ms SLO for Online Boutique across the dedicated and shared environments. Using models trained separately on data from each environment, AGAR predicts that meeting an SLO of 200 ms requires 36.6% more vCPUs on t3.2xlarge instances with shared tenancy than Dedicated Host tenancy.

The root cause is a misconfiguration chain: redis.conf sets no maxmemory bound, defaulting to Redis’s noeviction policy, so data accumulates with every write and memory grows monotonically. HPA, configured with a memory scaling threshold, mistakes this growth for load and scales the service out — providing only temporary relief as the dilution effect fades and memory resumes climbing, driving further scale-outs indefinitely. Notably, when HPA is configured to scale on CPU rather than memory with the same redis.conf, Redis accumulates data until the container’s memory limit is exhausted and the pod is OOMKilled. Setting maxmemory 100mb in redis.conf caps memory usage within the container’s 128Mi limit, and maxmemorypolicy allkeys-lru ensures Redis evicts the least recently used entries when the cap is reached rather than accumulating without bound — the specific policy matters less than having one. With these set, memory stabilizes once the working set fills the cap, eliminating the scale-out loop at its root.

5.4

Other Counterfactual Questions

5.4.1

SLO–cost trade-off.

5.4.2

Varying a Service’s Pod Startup Delay

The pod startup delay ∆C j (see §3.2) is used to generate the numerator of Equation 2: a longer startup window requires the autoscaler to provision for a higher peak RPS, since new pods cannot serve traffic until initialization completes. AGAR can therefore answer the counterfactual: how does reducing a service’s startup time affect cost? For TeaStore, this effect is specific to the service teastore -webui. Reducing its startup time from 30 s to 15 s lowers the cost of meeting a 300 ms SLO from 12 to 8 vCPUs/hr on t3.2xlarge instances with Dedicated Host tenancy and from 14 to 10 vCPUs/hr on instances with shared tenancy, corresponding to 33% and 29% reductions, respectively. The reduction comes from shrinking the RPS window over which the autoscaler must provision replicas. Applying the same optimization to other services yields no cost benefit, allowing developers to prioritize optimization effort.

Equation 2 makes the SLO target τ an explicit parameter, so AGAR can answer: given this workload, what SLO can I afford? To evaluate this, we sweep τ over a range of p95 latency targets and compute the hourly vCPU cost by replaying the observed RPS from an unseen trace through Equation 2 at each tick—without re-running the cluster. We validate these simulated estimates against live autoscaler runs on the same

5.5

Training Cost

Training time has two components. The first is generating candidate threshold configurations for the SLO satisfaction model in a threshold space of dimension d = (number of services) × (number of resource types). Although threshold search can in principle become expensive with d, threshold generation is 10

Env

App

H

500 1000 500 Shared TS 1000 500 OB 1000 500 SN 1000 500 Dedicated TS 1000 500 OB 1000 SN

6

Training time (s) Full Utilization bound retraining retraining 65.22 1.16–1.88 235.84 1.58–5.56 41.60 0.67–1.26 125.19 1.35–2.53 293.23 1.33–4.35 811.23 2.64–14.95 56.18 0.89–1.33 167.19 1.21–3.84 25.27 0.48–1.51 78.15 0.89–2.23 73.69 0.95–1.21 222.61 1.84–11.52

Related Work

Counterfactual performance modeling and what-if analysis. A related line of work uses causal or trace-driven models to answer hypothetical performance questions in complex systems. CausalSim [2] and Veritas [11] develop counterfactual reasoning methods for simulating networks. CausalSim learns system dynamics from randomized experiments to correct for the fact that an intervention can itself change the trace being replayed. Veritas in contrast avoids randomized experiments by constructing a domain-specific causal model that captures latent network conditions, trading experimental effort for modeling effort. Thesios [45] synthesizes representative and counterfactual full-resolution I/O traces from down-sampled storage traces, enabling what-if analyses of hypothetical policy and hardware changes. These systems share our interest in reasoning about unobserved scenarios, but they target different systems and do not compose per-component models. Our setting requires composition of several models. WebPerf [27] predicts request-latency distributions under hypothetical cloud configurations and workloads by combining request-dependency graphs with profiled cloud-API latencies, while AGAR reasons about SLO feasibility using per-service bounds learned from application measurements. Slowpoke [60] evaluates how optimizing a service affects application throughput by actively slowing down every other service to create an application with similar bottleneck structure, and avoids having to answer counterfactual questions from only passively collected data. Further, it answers a different question than AGAR. It focuses on a service’s throughput while we reason about its startup latency, among other things. TierSizer [61] uses analytical models to predict application slowdown at unexecuted DRAM sizes, enabling counterfactual DRAM sizing without instantiating those configurations, but like other work, this setting does not need models to compose. Autoscaling and resource management. A large body of work studies SLO-aware autoscaling and resource allocation for microservices. Most of it focuses on vertical rather than horizontal autoscaling; AGAR’s autoscaler targets the latter. These systems scale CPU and memory resources within a VM, rather than the number of VMs. Further, AGAR is more than an autoscaler. It can determine whether an SLO can be guaranteed under a given workload, what provisioning that guarantee requires, what it costs, and when scaling cannot help because the bottleneck is structural. Systems such as Sinan [63] and FIRM [47] use telemetrydriven machine learning to identify bottlenecks and vertically adjust per-service resources to maintain end-to-end latency targets. FIRM primarily performs vertical scaling, using scaleout only under resource oversubscription. AutoThrottle [57] similarly uses vertical scaling to enable online adaptation of per-service resources to application-level SLOs. It avoids the combinatorial cost of jointly optimizing per-service re-

Table 1: Full retraining, which is equivalent to initial training, reports the median of three runs, while utilization-bound retraining reports the range of median latencies across utilization models. Shared and Dedicated denote EC2 tenancy, and SN, TS, and OB denote Social Network, TeaStore, and Online Boutique, respectively. H is the number of training samples.

not the bottleneck in our implementation. This stage performs dominance filtering over observed SLO-violating utilization points followed by a cost-aware greedy search. The dominant cost arises when AGAR trains utilization models requested by this search. These fits are performed on demand, potentially in multiple batches at the same SLO, and cached by service, resource type, and threshold across the SLO sweep. For each threshold configuration, AGAR solves two LPs per service-resource pair: the first minimizes the transformed cost objective (Equation 2), and the second fixes that optimum and minimizes εi . The size of each LP is governed primarily by the number of endpoints and the number of training samples (H), since the formulation introduces one αi, j variable for each timestep-endpoint pair, for a total of #endpoints × H such variables, together with the associated monotonicity constraints. In our implementation, each CBC solve is singlethreaded, and parallelism comes from the outer joblib [28] layer across independent model fits, using two workers for batches and direct execution for singleton fits. Retraining can either be: (1) full retraining, which repeats the training procedure above, or (2) utilization-bound retraining, which refits only the affected utilization model (described in §3.2). Table 1 summarizes training and retraining costs. Across all applications and environments, median full-training latency is at most 293.23 s at H = 500 and 811.23 s at H = 1000, while median utilization-bound retraining latency is at most 14.95 s, just below the AGAR autoscaler’s 15 s decision interval. 11

sources using a bi-level controller, where local controllers translate application-level throttle ratios into CPU allocations, at the cost of potentially missing more resource-efficient allocations outside its restricted search space. Erlang [49] jointly allocates resources across all microservices of an application to minimize cost while satisfying an end-to-end latency target. GRAF [43] similarly targets end-to-end latency, using a graph neural network to proactively allocate resources based on workload changes. More structured approaches such as Ursa [64] and Erms [39] use analytic or piecewiselinear models to map per-service behavior to resource allocations and SLO satisfaction. Ursa actively searches for perservice utilization thresholds by reducing resources under replayed workloads until latency degrades, while Erms profiles piecewise-linear latency models over workload, CPU allocation, and interference for individual microservices and their dependencies. Galileo [50] uses a queueing-theoretic model to derive performance robustness certificates for request-type tail latencies under bounded environmental perturbations, and uses these certificates to guide learned autoscaling and admission control. These systems are primarily designed to choose or optimize allocations online. Debugging and root-cause analysis. Distributed tracing systems like Jaeger [51] and Zipkin [55] can be used to, among other things, identify the service which is experiencing high latency. To mitigate the instrumentation cost of tracing, prior work has explored tomography [8]. Murphy [25] uses a Markov random field to identify candidate root causes of performance problems from monitored metrics and provides interpretable explanations. Sage [18] identifies the service which caused an SLO violation and fixes it by reallocating resources (including compute, memory, cache or network resources) or reducing contention. However, it only focuses on problems that can be fixed by adding more resources to the problem. By developing an interpretable and global model of the application, AGAR can identify situations where an application is structurally incapable of reliably satisfying an SLO through autoscaling alone. Another line of work has focused on negative provenance to diagnose correctness (as opposed to performance) issues in networked systems [13, 59]. Formal modeling and verification. Another line of work reasons about microservice applications through explicit formal models. Kivi [36] model-checks Kubernetes controllers and configurations by exploring controller/event interleavings. Anvil [53] verifies that controller implementations satisfy eventually stable reconciliation, a liveness property. Both target correctness, rather than performance properties. Performal [62] is closer in spirit to AGAR than prior verification work: like AGAR, it uses upper bounds and can surface performance bugs. However, its bounds concern the worstcase end-to-end latency of single-request distributed systems, which is proved symbolically from a formal model of the distributed algorithm and then related to real time using isolated component measurements. Recently, a line of work on

performance verification has used assumption-constrained worst-case models to find pathological behaviors in existing algorithms [7, 20, 30, 40] and synthesize new ones [1, 31, 42]. Relatedly, another line of work analyzes program executions to determine worst-case running time and memory access patterns [26, 44]. All of these use formal reasoning over detailed models of the algorithms used and are hence limited in scope. AGAR is more data-driven and can thus reason about end-to-end performance of a microservice application without requiring detailed models of its implementation.

7

Discussion and Limitations

AGAR assumes that the fundamental behavior of the application remains stable over time. In deployment, we therefore recommend retraining the models after each code change and whenever the underlying hardware changes. In settings with substantial measurement noise or outliers, operators may choose to introduce slack in the models. Our framework can incorporate slack in all three models, allowing a small fraction of data points to violate the learned bounds. In our evaluation, slack was unnecessary for the utilization and SLO satisfaction models, as the applications were wellbehaved. However, if this assumption does not hold, slack can be introduced with minimal changes, as our implementation already supports it. Future work could extend AGAR to support vertical autoscaling and dynamic resource reallocation. More broadly, AGAR’s approach could apply to other systems problems where component-level relationships can be learned and composed to answer end-to-end counterfactual questions. Supporting richer models will require better abstractions and training methods for specifying and learning worst-case system behavior. LLMs could assist in constructing such models, but the resulting relationships must still provide explicit bounds that can be validated and composed. While composing AGAR models to reason about end-to-end guarantees was straightforward in our setting, more complex systems may require automated support. Developing general-purpose languages and algorithms for specifying and training such models is an important direction for future work.

8

Conclusion

We presented AGAR, a counterfactual performance analysis tool for microservice applications. AGAR builds simple models with explicit uncertainty bounds from readily available telemetry to reason about SLO feasibility under hypothetical changes to workloads, resource allocations, and SLO targets. We demonstrate how autoscaling and performance diagnosis can both be formulated as counterfactual queries that provide actionable guidance to operators. 12

References

Systems Design and Implementation (NSDI 26), pages 2011–2026, Renton, WA, May 2026. USENIX Association.

[1] Anup Agarwal, Venkat Arun, Devdeep Ray, Ruben Martins, and Srinivasan Seshan. Towards provably performant congestion control. In USENIX NSDI, 2024.

[10] Romil Bhardwaj, Kirthevasan Kandasamy, Asim Biswal, Wenshuo Guo, Benjamin Hindman, Joseph Gonzalez, Michael Jordan, and Ion Stoica. Cilantro: PerformanceAware resource allocation for general objectives via online feedback. In 17th USENIX Symposium on Operating Systems Design and Implementation (OSDI 23), pages 623–643, Boston, MA, July 2023. USENIX Association.

[2] Abdullah Alomar, Pouya Hamadanian, Arash NasrEsfahany, Anish Agarwal, Mohammad Alizadeh, and Devavrat Shah. CausalSim: A causal framework for unbiased Trace-Driven simulation. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pages 1115–1147, Boston, MA, April 2023. USENIX Association.

[11] Chandan Bothra, Jianfei Gao, Sanjay Rao, and Bruno Ribeiro. Veritas: Answering causal queries from video streaming traces. In Proceedings of the ACM SIGCOMM 2023 Conference, ACM SIGCOMM ’23, pages 738–753, New York, NY, USA, 2023. Association for Computing Machinery.

[3] Amazon Web Services. Amazon EC2 Dedicated Hosts. https://aws.amazon.com/ec2/dedicatedhosts/. Accessed: 2026-09-01. [4] Amazon Web Services. The EC2 approach to preventing side-channels. https://docs.aws.amazon.com/ whitepapers/latest/security-design-ofaws-nitro-system/the-ec2-approach-topreventing-side-channels.html.

[12] Abraham Charnes and William W. Cooper. Programming with linear fractional functionals. Naval Research Logistics Quarterly, 9(3–4):181–186, 1962.

[5] Amazon Web Services. Key concepts for burstable performance instances. https://docs.aws.amazon.com/ AWSEC2/latest/UserGuide/burstable-creditsbaseline-concepts.html.

[13] Ang Chen, Yang Wu, Andreas Haeberlen, Wenchao Zhou, and Boon Thau Loo. The good, the bad, and the differences: Better network diagnostics with differential provenance. In 2016 ACM SIGCOMM Conference (SIGCOMM ’16), August 2016.

[6] Amazon Web Services. Target tracking scaling policies for Amazon EC2 auto scaling. https://docs.aws.amazon.com/autoscaling/ec2/ userguide/as-scaling-target-tracking.html, 2024. Accessed: 2025-06-20.

[14] V. Chvatal. A greedy heuristic for the set-covering problem. Math. Oper. Res., 4(3):233–235, August 1979. https://doi.org/10.1287/moor.4.3.233. [15] Configuring Kubernetes. Liveness, Readiness and Startup Probes. https://kubernetes.io/docs/ tasks/configure-pod-container/configureliveness-readiness-startup-probes/. Accessed: 2025-07-08.

[7] Venkat Arun, Mina T. Arashloo, Ahmed Saeed, Mohammad Alizadeh, and Hari Balakrishnan. Toward formally verifying congestion control behavior. In ACM SIGCOMM, 2021.

[16] Datadog. State of containers and serverless. https://www.datadoghq.com/state-ofcontainers-and-serverless/, 2026. Accessed: 2026-09-03.

[8] Sachin Ashok, Vipul Harsh, Brighten Godfrey, Radhika Mittal, Srinivasan Parthasarathy, and Larisa Shwartz. Traceweaver: Distributed request tracing for microservices without application modification. In Proceedings of the ACM SIGCOMM 2024 Conference, pages 828– 842, 2024.

[17] Dmitry Duplyakin, Robert Ricci, Aleksander Maricq, Gary Wong, Jonathon Duerig, Eric Eide, Leigh Stoller, Mike Hibler, David Johnson, Kirk Webb, Aditya Akella, Kuangching Wang, Glenn Ricart, Larry Landweber, Chip Elliott, Michael Zink, Emmanuel Cecchet, Snigdhaswin Kar, and Prabodh Mishra. The design and operation of CloudLab. In Proceedings of the USENIX Annual Technical Conference (ATC), pages 1–14, July 2019.

[9] Mayank Bansal, Milind Chabbi, Kenneth Bøgh, Srikanth Prodduturi, Kevin Xu, Amit Kumar, David Bell, Ranjib Dey, Yufei Ren, Sachin Sharma, Juan Marcano, Shriniket Kale, Subhav Pradhan, Ivan Beschastnikh, Miguel Covarrubias, Chien-Chih Liao, Sandeep Koushik Sheshadri, Wen Luo, Kai Song, Ashish Samant, Sahil Rihan, Nimish Sheth, Albert Greenberg, and Uday Kiran Medisetty. Uber’s failover architecture: Reconciling reliability and efficiency in hyperscale microservice infrastructure. In 23rd USENIX Symposium on Networked

[18] Yu Gan, Mingyu Liang, Sundar Dev, David Lo, and Christina Delimitrou. Sage: Practical and scalable mldriven performance debugging in microservices. In 13

[27] Yurong Jiang, Lenin Ravindranath Sivalingam, Suman Nath, and Ramesh Govindan. Webperf: Evaluating "what-if" scenarios for cloud-hosted web applications. Technical Report MSR-TR-2016-16, June 2016.

Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’21, pages 135–151, New York, NY, USA, 2021. Association for Computing Machinery.

[28] Joblib Development Team. Joblib: running python functions as pipeline jobs, 2020. https:// joblib.readthedocs.io/.

[19] Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayan Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, Kelvin Hu, Meghna Pancholi, Yuan He, Brett Clancy, Chris Colen, Fukang Wen, Catherine Leung, Siyuan Wang, Leon Zaruvinsky, Mateo Espinosa, Rick Lin, Zhongling Liu, Jake Padilla, and Christina Delimitrou. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’19, page 3–18, New York, NY, USA, 2019. Association for Computing Machinery.

[29] Chris Jones, John Wilkes, Niall Murphy, and Cody Smith. Service level objectives. In Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy, editors, Site Reliability Engineering: How Google Runs Production Systems, chapter 4, pages 37–48. O’Reilly Media, 2016. [30] Pantea Karimi, Solal Pirelli, Siva Kesava Reddy Kakarla, Ryan Beckett, Santiago Segarra, Beibin Li, Pooria Namyar, and Behnaz Arzani. Towards safer heuristics with xplain. In Proceedings of the 23rd ACM Workshop on Hot Topics in Networks, pages 68–76, 2024.

[20] Saksham Goel, Benjamin Mikek, Jehad Aly, Venkat Arun, Ahmed Saeed, and Aditya Akella. Quantitative verification of scheduling heuristics. arXiv, 2023.

[31] Pantea Karimi, Dany Rouhana, Pooria Namyar, Siva Kesava Reddy Kakarla, Venkat Arun, and Behnaz Arzani. Robust heuristic algorithm design with llms. arXiv preprint arXiv:2510.08755, 2025.

[21] Google Cloud. Autoscaling groups of instances. https: //cloud.google.com/compute/docs/autoscaler, 2024. Accessed: 2025-06-20.

[32] Richard M. Karp. Reducibility among combinatorial problems. In Raymond E. Miller and James W. Thatcher, editors, Complexity of Computer Computations, pages 85–103, New York, 1972. Plenum Press.

[22] Google Cloud. About autoscaling workloads based on metrics. https://docs.cloud.google.com/ kubernetes-engine/docs/concepts/custom-andexternal-metrics, 2026. Google Kubernetes Engine documentation, last updated 2026-03-26, accessed 2026-03-30.

[33] Knative Project. Autoscaling concepts. https: //knative.dev/docs/serving/autoscaling/ autoscaling-concepts/, 2024. Accessed: 2025-0620.

[23] GoogleCloudPlatform. Online Boutique: A cloud-native microservices demo application. https://github.com/GoogleCloudPlatform/ microservices-demo, 2020.

[34] Kubernetes. Horizontal pod autoscaling. https: //kubernetes.io/docs/concepts/workloads/ autoscaling/horizontal-pod-autoscale/. Accessed: 2026-09-17.

[24] Adam Hall, Anirudh Sarma, Esha Choukse, Umakishore Ramachandran, and Sameh Elnikety. Harvesting spare CPU resources in container systems. In 23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI 26), pages 2323–2337, Renton, WA, May 2026. USENIX Association.

[35] Kubernetes. Horizontal pod autoscaler. https://kubernetes.io/docs/tasks/runapplication/horizontal-pod-autoscale/, 2024. Accessed: 2025-06-20.

[25] Vipul Harsh, Wenxuan Zhou, Sachin Ashok, Radhika Niranjan Mysore, Brighten Godfrey, and Sujata Banerjee. Murphy: Performance diagnosis of distributed cloud applications. In Proceedings of the ACM SIGCOMM 2023 Conference, pages 438–451, 2023.

[36] Bingzhe Liu, Gangmuk Lim, Ryan Beckett, and P. Brighten Godfrey. Kivi: Verification for cluster management. In 2024 USENIX Annual Technical Conference (USENIX ATC 24), pages 509–527, Santa Clara, CA, July 2024. USENIX Association.

[26] Rishabh Iyer, Luis Pedrosa, Arseniy Zaostrovnykh, Solal Pirelli, Katerina Argyraki, and George Candea. Performance contracts for software network functions. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19), pages 517–530, 2019.

[37] Locust team. Locust: A modern load testing framework, ongoing. https://locust.io/. [38] Shutian Luo, Huanle Xu, Chengzhi Lu, Kejiang Ye, Guoyao Xu, Liping Zhang, Yu Ding, Jian He, and 14

Chengzhong Xu. Characterizing microservice dependency and performance: Alibaba trace analysis. In Proceedings of the ACM Symposium on Cloud Computing, pages 412–426, 2021.

[47] Haoran Qiu, Subho S. Banerjee, Saurabh Jha, Zbigniew T. Kalbarczyk, and Ravishankar K. Iyer. FIRM: An intelligent fine-grained resource management framework for SLO-Oriented microservices. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pages 805–825. USENIX Association, November 2020.

[39] Shutian Luo, Huanle Xu, Kejiang Ye, Guoyao Xu, Liping Zhang, Jian He, Guodong Yang, and Chengzhong Xu. Erms: Efficient resource management for shared microservices with sla guarantees. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, ASPLOS 2023, page 62–77, New York, NY, USA, 2022. Association for Computing Machinery.

[48] Krzysztof Rzadca, Pawel Findeisen, Jacek Swiderski, Przemyslaw Zych, Przemyslaw Broniek, Jarek Kusmierek, Pawel Nowak, Beata Strack, Piotr Witusowski, Steven Hand, and John Wilkes. Autopilot: workload autoscaling at google. In Proceedings of the Fifteenth European Conference on Computer Systems, EuroSys ’20, New York, NY, USA, 2020. Association for Computing Machinery.

[40] Pooria Namyar, Behnaz Arzani, Ryan Beckett, Santiago Segarra, Himanshu Raj, Umesh Krishnaswamy, Ramesh Govindan, and Srikanth Kandula. Finding adversarial inputs for heuristics using multi-level optimization. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pages 927–949, 2024.

[49] Vighnesh Sachidananda and Anirudh Sivaraman. Erlang: Application-aware autoscaling for cloud microservices. In Proceedings of the Nineteenth European Conference on Computer Systems, EuroSys ’24, page 888–923, New York, NY, USA, 2024. Association for Computing Machinery.

[41] Oracle. JVM tuning. https://docs.oracle.com/cd/ E11035_01/wls100/perform/JVMTuning.html. Oracle WebLogic Server 10.0 Performance and Tuning.

[50] Divyanshu Saxena, Gaurav Vipat, Jiaxin Lin, Jingbo Wang, Isil Dillig, Sanjay Shakkottai, and Aditya Akella. Towards performance robustness for microservices. In 23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI 26), pages 93–112, Renton, WA, May 2026. USENIX Association.

[42] Tony Pan, Anup Agarwal, Isil Dillig, and Venkat Arun. Syntra: Synthesizing cross-layer controllers for lowlatency video streaming. In USENIX NSDI, 2026. [43] Jinwoo Park, Byungkwon Choi, Chunghan Lee, and Dongsu Han. Graf: a graph neural network based proactive resource allocation framework for slo-oriented microservices. In Proceedings of the 17th International Conference on Emerging Networking EXperiments and Technologies, CoNEXT ’21, page 154–167, New York, NY, USA, 2021. Association for Computing Machinery.

[51] Yuri Shkuro. Evolving distributed tracing at uber engineering. https://www.uber.com/blog/ distributed-tracing/, 2017. Accessed: 2026-0401.

[44] Luis Pedrosa, Rishabh Iyer, Arseniy Zaostrovnykh, Jonas Fietz, and Katerina Argyraki. Automated synthesis of adversarial workloads for network functions. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pages 372–385, 2018.

[52] Gagan Somashekar, Karan Tandon, Anush Kini, ChiehChun Chang, Petr Husak, Ranjita Bhagwan, Mayukh Das, Anshul Gandhi, and Nagarajan Natarajan. OPPerTune: Post-Deployment configuration tuning of services made easy. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pages 1101–1120, Santa Clara, CA, April 2024. USENIX Association.

[45] Phitchaya Mangpo Phothilimthana, Saurabh Kadekodi, Soroush Ghodrati, Selene Moon, and Martin Maas. Thesios: Synthesizing accurate counterfactual i/o traces from i/o samples. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, ASPLOS ’24, page 1016–1032, New York, NY, USA, 2024. Association for Computing Machinery.

[53] Xudong Sun, Wenjie Ma, Jiawei Tyler Gu, Zicheng Ma, Tej Chajed, Jon Howell, Andrea Lattuada, Oded Padon, Lalith Suresh, Adriana Szekeres, and Tianyin Xu. Anvil: Verifying liveness of cluster management controllers. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pages 649–666, Santa Clara, CA, July 2024. USENIX Association.

[46] PuLP. Pulp: A linear programming toolkit for python. https://coin-or.github.io/pulp/. Accessed: 2025-07-08.

[54] Steven Thurgood, David Ferguson, Alex Hidalgo, and Betsy Beyer. Implementing SLOs, 2018. https:// sre.google/workbook/implementing-slos/. 15

[55] Twitter, Inc. Zipkin: A distributed tracing system. https://zipkin.io/, 2012. Accessed: 2026-04-01.

Languages and Operating Systems, ASPLOS ’21, page 167–181, New York, NY, USA, 2021. Association for Computing Machinery.

[56] Jóakim von Kistowski, Simon Eismann, Norbert Schmitt, André Bauer, Johannes Grohmann, and Samuel Kounev. TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research. In Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems, MASCOTS ’18, September 2018.

[64] Yanqi Zhang, Zhuangzhuang Zhou, Sameh Elnikety, and Christina Delimitrou. Ursa: Lightweight Resource Management for Cloud-Native Microservices . In 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 954–969, Los Alamitos, CA, USA, March 2024. IEEE Computer Society. [65] Zhizhou Zhang, Murali Krishna Ramanathan, Prithvi Raj, Abhishek Parwal, Timothy Sherwood, and Milind Chabbi. CRISP: Critical path analysis of Large-Scale microservice architectures. In 2022 USENIX Annual Technical Conference (USENIX ATC 22), pages 655– 672, Carlsbad, CA, July 2022. USENIX Association.

[57] Zibo Wang, Pinghe Li, Chieh-Jan Mike Liang, Feng Wu, and Francis Y. Yan. Autothrottle: A practical Bi-Level approach to resource management for SLO-Targeted microservices. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pages 149–165, Santa Clara, CA, April 2024. USENIX Association. [58] Yang Wu, Ang Chen, and Linh Thi Xuan Phan. Zeno: Diagnosing performance problems with temporal provenance. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19), pages 395– 420, Boston, MA, February 2019. USENIX Association. [59] Yang Wu, Mingchen Zhao, Andreas Haeberlen, Wenchao Zhou, and Boon Thau Loo. Diagnosing missing events in distributed systems with negative provenance. In Proceedings of the 2014 ACM Conference on SIGCOMM, SIGCOMM ’14, page 383–394, New York, NY, USA, 2014. Association for Computing Machinery. [60] Yizheng Xie, Di Jin, Oğuzhan Çölkesen, Vasiliki Kalavri, John Liagouris, and Nikos Vasilakis. Slowpoke: End-to-end throughput optimization modeling for microservice applications. In 23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI 26), pages 631–646, Renton, WA, May 2026. USENIX Association. [61] Sujay Yadalam, Saarth Deshpande, Divyanshu Saxena, Venkat Arun, Neeraja J. Yadwadkar, and Aditya Akella. TierSizer: Counterfactual reasoning for DRAM sizing in tiered memory systems. In 4th Workshop on Disruptive Memory Systems (DIMES), 2026. [62] Tony Nuda Zhang, Upamanyu Sharma, and Manos Kapritsos. Performal: Formal verification of latency properties for distributed systems. Proc. ACM Program. Lang., 7(PLDI), June 2023. [63] Yanqi Zhang, Weizhe Hua, Zhuangzhuang Zhou, G. Edward Suh, and Christina Delimitrou. Sinan: Ml-based and qos-aware resource management for cloud microservices. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming 16

A

Utilization model optimization

Algorithm 1 Greedy SLO-safe threshold selection Input: Violations R; threshold sets T j ; replica costs f j ; service dimensions Js Output: SLO-safe thresholds v, or infeasible 1: v j ← max T j for every dimension j 2: R ← {x ∈ R : ∀ j, x j < v j } 3: Bs ← { f j (t) : j ∈ Js , t ∈ T j } for every service s 4: Cs (z) = max j∈Js f j (z j ) for every service s 5: while R ̸= ∅ do 6: ρ ← ∞; best ← none 7: for all services s do 8: for all b ∈ Bs : b ≥ Cs (v) do 9: u←v 10: for all j ∈ Js do 11: u j ← min{t ∈ T j : t ≤ v j , f j (t) ≤ b}

Since the objective in Equation 2 is not linear in the LP variables βi and εi , the objective is linear-fractional and can be converted into an LP using a standard Charnes–Cooper transformation [12]. We introduce an auxiliary variable Xi with constraints Xi ≥ X ∑ j αi, j (rps j ), so that the objective becomes Ti−βii −εi . We then define λi = Ti −β1i −εi and scaled variables α̂i, j (·) = λi αi, j (·), βˆ i = λi βi , ε̂i = λi εi , and X̂i = λi Xi . This yields the normalization constraint Ti λi − βˆ i − ε̂i = 1 and turns the objective into minimizing X̂i . The constraints Xi ≥ ∑ j αi, j (rps j ) become X̂i ≥ ∑ j α̂i, j (rps j ), which are linear. The utilization-fit and monotonicity constraints remain linear under the same change of variables. Thus, for each candidate threshold Ti , training reduces to solving LPs rather than a nonlinear program. In practice, we solve two LPs in sequence: the first minimizes the transformed cost objective, and the second fixes that optimum while minimizing ε̂i to prefer tighter uncertainty bounds among equally cost-effective solutions.

B

E ← {x ∈ R : ∃ j ∈ Js , x j ≥ u j } if E ̸= ∅ then ∆ ← Cs (u) −Cs (v) if ∆/|E| < ρ then ρ ← ∆/|E| best ← (u, E) 18: if best = none then 19: return infeasible 20: (v, E) ← best 21: R ← R\E 22: return v 12: 13: 14: 15: 16: 17:

SLO-safe threshold selection

We reduce set cover [32] to threshold selection. Consider an arbitrary set-cover instance with universe U = {u1 , . . . , un } and sets S = {S1 , . . . , Sd }. We construct a threshold-selection instance with one utilization threshold for each set S j and one SLO-violating observation for each element ui . Thus, each observation has d utilization values, one for each service. An observation is excluded if, for at least one service, its observed utilization is greater than or equal to that service’s threshold. Choose constants 0 ≤ a < b < 99. For the observation representing ui , its utilization in dimension j is b if ui ∈ S j and a otherwise. Therefore, lowering threshold j from 99 to b excludes exactly the violations representing the elements in S j . In other words, lowering threshold j has the same effect as selecting set S j . Define each threshold’s additional replica cost as 0 above b, 1 at b, and d + 1 below b. These costs are non-increasing in the threshold. Lowering all d thresholds to b excludes every observation at an additional cost of d, so no optimal solution lowers a threshold below b. Any threshold above b can be replaced by 99 without changing its cost or which observations it excludes. Consequently, lowering k thresholds to b excludes every violation exactly when their corresponding k sets cover U, at a cost of k additional replicas. Minimizing this cost therefore finds a minimum set cover. The construction has polynomial size, proving that threshold selection with general monotone costs is NP-hard.

ues below 99, T j the finite nonempty set of representative valid thresholds for utilization dimension j, and f j (t) the replica requirement obtained by fitting the utilization model at threshold t. For each dimension, thresholds excluding the same observations in R are represented by their largest valid original candidate. Each f j is non-increasing. For service s, let Js contain its utilization dimensions (CPU and memory). Its replica requirement is Cs (v) = max f j (v j ). j∈Js

We consider service-level replica budgets drawn from

Bs = { f j (t) : j ∈ Js , t ∈ T j }. For each budget, we lower that service’s thresholds as far as the budget permits. Adapting the greedy set-cover heuristic [14], we choose the move with the smallest additional replica cost per newly excluded violation, repeating until all violations are excluded (Algorithm 1). An observation is excluded when its utilization in any dimension is greater than or equal to that dimension’s threshold.

C

Greedy algorithm. Let R be the distinct, componentwiseminimal SLO-violating observations with all utilization val-

Evaluation Workloads

Figure 10 shows representative 6–8 h segments from the Alibaba-derived traces used during evaluation. These seg17

ments are the intervals on which we evaluate autoscaler behavior after training on the preceding 6 hours. The traces exhibit substantial diversity: some are nearly flat, some show small bursty fluctuations, and others contain sharp spikes or sustained high load. During replay, we assign one such trace to each application endpoint, so different endpoints within the same application may experience qualitatively different demand patterns over the evaluation window.

Median p95 latency (ms)

HPA

Erlang

AGAR

1.2K 900 600 300 0

5

10

20

30

40

Median p95 latency (ms)

Replicas (a) Online Boutique 60K 900 600 300 0

5

10

20

30

40

30

40

Median p95 latency (ms)

Replicas (b) TeaStore 1.2K 900 600 300 0

5

10

20

Replicas (c) Social Network

Figure 10: Representative replay workloads used in evaluation. Each panel shows the 6–8 h evaluation segment of a 12 h Alibaba-derived request-rate trace. One such trace is assigned to each application endpoint during replay.

Figure 11: Per-endpoint median p95 latency when using HPA, Erlang, and AGAR on CloudLab. The horizontal dashed line marks the 300 ms SLO. Hollow markers indicate unfixed configurations (described in §5.3). For each application, the boundary boxes mark the average replica count for each autoscaler configuration.

E Effect of Host Interference on AGAR’s Models D

Evaluating AGAR on CloudLab

We run all experiments on a CloudLab [17] cluster of r320 machines running Kubernetes with kubeadm. AGAR’s Autoscaler runs on a dedicated node on which Kubernetes pod scheduling is disabled, so the autoscaler itself does not contend for resources the application needs.

To examine how host-level interference is reflected in AGAR’s learned models, Tables 2–4 compare the fitted utilizationmodel parameters on dedicated and shared hosts for a 300 ms 18

Service adservice cartservice checkoutservice currencyservice emailservice frontend paymentservice productcatalogservice recommendationservice redis-cart shippingservice

Metric cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory

β (Dedicated) 6.0 – 10.5 57.0 2.0 11.5 16.0 0.0 1.0 66.0 53.0 22.5 1.0 – 29.0 14.5 22.5 64.5 2.5 7.0 4.0 11.5

β (Shared) 19.5 – 14.5 64.5 4.0 13.5 19.5 – 1.5 66.0 0.0 24.5 2.0 – 50.0 15.0 44.5 65.5 8.0 8.0 5.5 12.5

ε (Dedicated) 4.0 – 4.5 9.0 2.0 1.5 16.0 50.0 1.0 1.0 27.0 4.5 1.0 – 14.0 1.5 13.5 0.5 1.5 0.0 2.0 1.5

ε (Shared) 16.5 – 5.5 12.5 4.0 1.5 19.5 – 1.5 1.0 24.1 3.5 2.0 – 24.0 2.0 27.5 1.5 6.0 1.0 2.5 1.5

T (Dedicated) 99.0 – 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 – 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0

T (Shared) 99.0 – 99.0 99.0 99.0 99.0 99.0 – 99.0 99.0 99.0 99.0 99.0 – 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0

Table 2: Learned utilization-model parameters for Online Boutique on EC2 t3.2xlarge instances with dedicated and shared tenancy. Service teastore-auth teastore-db teastore-image teastore-persistence teastore-recommender teastore-registry teastore-webui

Metric cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory

β (Dedicated) 24.5 29.0 3.0 3.0 39.5 44.5 35.0 33.5 0.0 31.0 6.0 17.5 0.0 58.0

β (Shared) 51.0 29.5 5.5 3.5 – 50.0 0.0 0.0 26.0 46.5 5.0 17.0 27.5 58.5

ε (Dedicated) 10.5 1.0 2.0 0.0 17.5 2.5 14.0 2.5 11.5 2.0 6.0 5.5 42.0 18.0

ε (Shared) 35.0 1.5 3.5 0.5 – 2.0 53.7 17.7 17.0 2.5 5.0 5.0 37.5 18.5

T (Dedicated) 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 27.0 99.0 99.0 99.0 99.0 99.0

T (Shared) 99.0 99.0 99.0 99.0 – 99.0 99.0 43.0 99.0 99.0 99.0 99.0 99.0 99.0

Table 3: Learned utilization-model parameters for TeaStore on EC2 t3.2xlarge instances with dedicated and shared tenancy. p95 latency SLO. Differences in β capture shifts in baseline utilization, while differences in ε capture changes in unexplained variability; threshold changes reflect corresponding changes in the utilization regions deemed SLO-safe.

19

Service compose-post-service home-timeline-redis home-timeline-service media-frontend media-memcached media-service nginx-thrift post-storage-memcached post-storage-service social-graph-redis social-graph-service text-service unique-id-service url-shorten-memcached url-shorten-service user-memcached user-mention-service user-service user-timeline-redis user-timeline-service

Metric cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory cpu memory

β (Dedicated) 48.5 9.0 2.5 3.0 10.0 5.0 0.0 21.0 0.0 1.0 4.0 1.0 43.0 63.0 17.0 – 40.5 6.5 4.0 3.0 14.0 3.0 27.0 3.0 3.5 1.0 0.0 1.0 12.0 3.5 0.0 2.0 15.5 3.5 3.5 2.0 13.5 0.0 27.5 37.0

β (Shared) 0.0 9.0 3.0 3.0 24.5 6.5 0.0 21.0 0.0 1.0 7.5 1.5 47.0 66.0 27.5 0.0 43.0 9.5 8.5 3.0 29.5 5.0 52.0 5.0 8.5 1.5 0.0 1.0 28.5 4.0 19.5 2.0 29.5 3.5 7.0 3.0 31.0 0.0 29.0 –

ε (Dedicated) 8.5 2.0 1.5 0.0 4.0 2.0 0.0 0.0 0.0 0.0 2.0 0.0 29.0 8.0 16.0 – 30.5 2.5 1.0 0.0 6.0 0.0 11.0 1.0 2.5 0.0 0.0 0.0 7.0 0.5 2.0 1.0 6.5 0.5 2.5 1.0 7.5 48.5 13.5 31.0

ε (Shared) 17.3 4.0 2.0 0.0 17.5 2.5 0.0 0.0 0.0 0.0 5.5 0.5 32.0 10.0 24.5 62.0 33.0 5.5 4.5 0.0 15.5 2.0 34.0 2.0 5.5 0.5 0.0 0.0 15.5 1.0 9.5 1.0 15.5 0.5 4.0 0.0 26.0 65.3 29.0 –

T (Dedicated) 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 – 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 10.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0

T (Shared) 39.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 99.0 –

Table 4: Learned utilization-model parameters for Social Network on EC2 t3.2xlarge instances with dedicated and shared tenancy.

20

Related documents

• Agar: What-If Analysis for Microservice Applications#1467960
• OSWorld-Pro: Process-based Evaluation for Computer Use Agents#1051287
• OSWorld-Pro: Process-based Evaluation for Computer Use Agents#1028686
• TokenBank: Financial Infrastructure for AI Services#1248613
• Building Microservices#1190754