A Unified Queueing-Based Framework for Comparative Evaluation of Serverless Autoscaling Strategies under Bursty Workloads

A Unified Queueing-Based Framework for Comparative Evaluation of Serverless Autoscaling Strategies under Bursty Workloads Zahra Zafarzadea , Ehsan Ataiea,∗

arXiv:2610.35564v1 [cs.DC] 26 Oct 2026

a Department of Computer Engineering, University of Mazandaran, Babolsar, Iran

ARTICLE INFO

ABSTRACT

Keywords: Serverless Computing Autoscaling Queueing Petri Nets Performance Modeling

Serverless computing platforms must preserve quality of service (QoS) while adapting to highly bursty and unpredictable workloads. To cope with workload fluctuations while satisfying QoS requirements and service-level agreements (SLAs), these platforms rely on automatic resource scaling. Commercial function-as-a-service (FaaS) platforms employ different autoscaling strategies to dynamically adjust the number of active function instances and the available processing capacity. In this work, three representative autoscaling strategies are modeled and compared: scale-per-request (SPR), concurrencylevel scaling (CLS), and metric-based scaling (MBS). To enable systematic evaluation independent of proprietary platform-specific implementations, we develop a unified modeling framework that combines analytical queueing network (QN) modeling with queueing Petri nets (QPN) implemented in Java Modeling Tools (JMT) to evaluate autoscaling behavior under two-phase hyper-exponential (𝐻2 ) arrivals. Evaluation is conducted using four system-level metrics: throughput, mean response time, system population, and system power. The framework captures dynamic function instance (FI) activation, concurrency constraints, and queue buildup under saturation. Comparative analysis reveals a clear trade-off between peak performance and stability: SPR provides the lowest delay under moderate load but becomes fragile near capacity limits; CLS is governed by concurrency tokens and is highly sensitive to configuration under bursty conditions; whereas MBS provides the most stable QoS across the workload range by keeping utilization below a critical region and preventing explosive queue growth under critical load.

1. Introduction Serverless computing has emerged as a major paradigm in cloud systems. Under the function-as-a-service (FaaS) model, services are executed in an event-driven manner and deployed without direct management of the underlying infrastructure [1, 2]. To execute these functions, the platform creates and manages isolated execution environments, referred to in this study as function instances (FIs). An FI provides the runtime environment and the computational resources required to process function invocations [3, 4]. Depending on the platform architecture, these execution environments may be implemented using containers, virtual machines, or lightweight microVM-based isolation mechanisms [3, 5, 6]. These implementation details are typically hidden from the user, while the FI acts as the practical unit through which processing capacity is provided and adjusted. Resource provisioning is therefore handled by the platform, which reacts to workload variation by activating, reusing, or retiring FIs. Depending on the platform and its concurrency configuration, an FI may process a single request or multiple concurrent requests. While this abstraction reduces operational burden, it also makes quality of service (QoS), especially under dynamic demand, strongly dependent on the design and tuning of autoscaling mechanisms [1, 2, 7]. In real deployments, request streams are often bursty and difficult to anticipate, and such variability can amplify ∗ Corresponding author

z.zafarzade11@umail.umz.ac.ir (Z. Zafarzade); ataie@umz.ac.ir

queueing effects as systems approach saturation [7, 8, 9]. FaaS platforms must therefore increase capacity within short time windows while facing practical constraints such as instance activation delays, per-instance concurrency limits, and finite resources [10, 11]. Under such conditions, delayed or unstable scaling decisions may prevent FaaS platforms from responding effectively to workload fluctuations. This limitation can drive the system toward sustained high utilization, where cold-start overheads, resource contention, and queue accumulation become more pronounced, ultimately resulting in degraded user-perceived latency [12, 13, 9]. Existing studies have investigated autoscaling and performance modeling for serverless and cloud-native platforms using analytical methods, simulation, and measurementbased evaluation [1, 14]. However, comparing different autoscaling strategies under identical conditions remains challenging because workload characteristics and platform constraints can strongly affect their observed behavior. In addition, commercial FaaS platforms provide limited visibility into their internal scaling mechanisms, making detailed analysis and independent validation more difficult [15]. To address this gap, a unified modeling framework is presented for studying autoscaling behavior in FaaS platforms. Queueing network (QN) theory is combined with queueing petri nets (QPN) implemented in the java modeling tools (JMT), enabling a consistent representation of capacity adaptation and system behavior under increasing load. Bursty demand is represented using a two-phase hyperexponential (𝐻2 ) arrival process, allowing controlled evaluation under high-variability traffic. Within this framework,

(E. Ataie) ORCID (s):

Author et al.: Preprint submitted to Elsevier

Page 1 of 18

Autoscaling in Serverless Computing

three autoscaling families are modeled and compared: scaleper-request (SPR), concurrency-level scaling (CLS), and metric-based scaling (MBS). The main contributions of this work are summarized as follows: 1. A unified modeling and evaluation framework is developed for FaaS autoscaling strategies by combining QN models with QPN-based representations. This framework provides a transparent and reproducible foundation for analyzing autoscaling behavior. 2. The proposed QPN models explicitly represent operational constraints of FaaS platforms, including FI availability, per-instance concurrency limits, and queue accumulation under constrained processing capacity. 3. A burst-aware workload representation based on an 𝐻2 arrival process is incorporated to investigate autoscaling behavior under variable traffic conditions. 4. To the best of our knowledge, this work presents the first comprehensive analytical comparison of three FaaS autoscaling paradigms, including SPR, CLS, and MBS scaling, within a unified framework. Furthermore, the MBS strategy combines utilization monitoring phases with adaptive routing decisions in a threshold-driven metric-based scaling approach. 5. A QoS-oriented performance assessment approach is established to analyze the impact of autoscaling decisions on service quality. By considering throughput, mean response time, in-system population, and system power, the approach reveals the trade-offs between scaling responsiveness and system stability. The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 introduces the analytical modeling formalisms used in this study. Section 4 describes the system under study, including the workload characteristics and modeling assumptions. Section 5 presents the proposed autoscaling models and evaluation metrics. Section 6 reports and discusses the performance evaluation results. Finally, Section 7 concludes the paper and outlines directions for future work.

2. Related Work Many studies have investigated resource management, capacity planning, and performance optimization in microservices and serverless computing environments. These studies cover a wide range of autoscaling mechanisms and evaluation approaches, including empirical, controltheoretic, analytical, and stochastic modeling methods. To provide a structured overview, the related work is divided into two main groups. Section 2.1 reviews autoscaling mechanisms in microservices and serverless platforms, while Section 2.2 examines analytical and stochastic modeling approaches to autoscaling behavior and system performance.

Author et al.: Preprint submitted to Elsevier

2.1. Autoscaling in Serverless and Microservices Tran and Kim [16] proposed a proactive hybrid autoscaling method for quota-constrained Knative systems, combining traffic prediction and deep reinforcement learning for concurrent services sharing a resource quota. Their experiments report fewer latency SLO violations and improved quota compliance compared with conventional Knative autoscaling, highlighting the need to coordinate scaling decisions across services. Tran and Kim [17] developed a hybrid autoscaling approach for Knative-based serverless edge computing. It combines horizontal and vertical scaling by adjusting instance counts, CPU allocation, and request concurrency using predicted traffic and service profiles. The implementation extends Kubernetes operators without modifying Knative and reduces resource usage while maintaining latency SLOs, although it depends on accurate workload prediction and profiling. Shafi et al. [18] proposed CEMA, a predictive multilayer autoscaling framework for microservices. It dynamically adjusts container and VM counts according to workload demand and migrates containers from underutilized VMs to reduce resource waste and infrastructure cost. Evaluated using real-world workload traces, CEMA focuses on containerized resource management rather than analytical comparison of FaaS scaling strategies. Zafeiropoulos et al. [19] developed reinforcement learning (RL)-assisted autoscaling mechanisms based on Qlearning, DynaQ+, and deep Q-learning. The agents make scaling decisions using resource and QoS metrics and are evaluated in both Kubeless deployments and simulated environments. The results support RL-based adaptation under dynamic workloads, although more complex agents require substantial training. Khaleq and Ra [20] extended Kubernetes HPA with an RL-based autoscaling module for microservices. Their method selects CPU or memory metrics according to workload demand to improve response time. Experiments on Google Kubernetes Engine (GKE) with a Twitter analytics application show lower latency than default HPA, with the evaluation primarily addressing horizontal scaling. Benedetti et al. [12] proposed a Q-learning-based autoscaling method for serverless functions in edge computing. It dynamically adjusts HPA configurations using Azure invocation patterns to improve latency-based SLA compliance while limiting unnecessary resource allocation. Experimental results indicate lower resource consumption and fewer request losses than baseline HPA configurations. Mampage et al. [21] introduced an A3C-based multiagent RL approach that jointly optimizes horizontal and vertical scaling in multi-tenant serverless applications. The method balances application response time and infrastructure cost through configurable optimization priorities. Evaluations using Azure traces and a Kubeless-based Kubernetes testbed show that different priorities favor either lower latency or reduced cost.

Page 2 of 18

Autoscaling in Serverless Computing

Li et al. [22] presented EAES, an energy-aware elastic scaling algorithm for Kubernetes microservices that coordinates feedforward and feedback control. It improves container provisioning by releasing excess containers and accounting for energy consumed by idle resources. Experiments report a 15.34% reduction in cluster energy consumption compared with baseline scaling methods while maintaining SLA requirements. Abdullah et al. [23] proposed an RL-based autoscaling approach that dynamically selects heterogeneous VM types to balance application performance and infrastructure cost. Evaluated using five real-world workload traces, including ClarkNet, NASA, Wikipedia, Calgary, and FIFA World Cup, it processes 10.49% more requests on average and achieves 7.89× fewer SLO violations than the evaluated baselines.

2.2. Autoscaling with Analytical and Stochastic Modeling Ship et al. [24] developed an optimization-based serverless autoscaling method using a fluid approximation and Separated Continuous Linear Programming (SCLP). Their approach jointly determines per-replica resource allocation and the required number of replicas over time to reduce response times and failed requests. Python/SimPy-based simulations demonstrate improvements over threshold-based autoscaling under changing workloads. Zafarzade and Ataie [25] proposed an analytical capacity planning approach for a microservices-based image classification application using Extended Queueing Networks. By analyzing utilization and performance at the function-station and system levels, their model identifies bottlenecks and guides changes in the number of FIs to avoid overutilization under varying workloads. Wu et al. [26] introduced CloudNativeSim, a CloudSimbased toolkit for modeling and simulating cloud-native microservice applications. The framework represents service dependencies and supports configurable scheduling and scaling policies. Validation against a reference deployment reports over 94.5% accuracy in response-time simulation, demonstrating its suitability for evaluating alternative scaling policies. F’e et al. [27] proposed a stochastic Petri net (SPN) model for performance–cost analysis of cloud autoscaling. It captures VM instantiation and termination and evaluates throughput, response time, and provisioning cost across configurations. Combined with GRASP optimization, the model identifies cost-effective settings subject to SLA constraints and is validated against 18,000 real-testbed samples. Gias et al. [28] presented ATOM, a model-driven autoscaling framework for microservices based on runtime layered queueing network (LQN) models. It jointly adjusts replica counts and CPU shares, combining horizontal and vertical scaling. Under heavy workloads, the study reports 30–37% higher throughput than static-rule controllers, although its target environment is microservices rather than FaaS.

Author et al.: Preprint submitted to Elsevier

Luan and Shen [29] proposed an SLO-aware optimization method for containerized microservices combining layered queueing networks (LQNs), deep neural networks, and particle swarm optimization. Their approach seeks to reduce CPU resource costs while satisfying service-level objectives, with an emphasis on resource allocation in containerized applications. Gao et al. [30] developed analytical performance models for distributed data processing in microservices. The models account for data-access workflows, caching behavior, and queueing contention to evaluate performance and guide service/data placement. Their primary focus is performance prediction and placement optimization rather than runtime autoscaling decisions. Lim et al. [31] introduced S-iNAS, a stochastic reward net (SRN)-based framework for analyzing time- and eventbased scaling in distributed storage systems. Their results indicate that event-based scaling can reduce request latency compared with time-based policies, although it may trigger more frequent resource reconfiguration. The study addresses storage-layer rather than FaaS autoscaling. Ataie et al. [32] proposed a stochastic activity network (SAN)-based model for evaluating dispatching policies in IaaS cloud data centers. The model analyzes performance, availability, and power consumption and is validated against CloudSim. Comparisons across dispatching policies illustrate its analytical capabilities, while the primary focus remains resource dispatching rather than autoscaling control. Merkouche et al. [33] proposed a Petri net-based formal model for microservice autoscaling. The study explores the formal representation of microservice scaling behavior rather than a performance comparison of multiple serverless autoscaling strategies. Jawaddi et al. [34] investigated energy-aware Kubernetes microservice autoscaling through four Markov Decision Process (MDP) variants and probabilistic model checking. Their models incorporate scaling constraints inspired by Kubernetes HPA and support design-time evaluation of host energy consumption and service-level requirements under different scaling policies. Although previous studies have advanced autoscaling mechanisms and performance modeling, these topics are often investigated separately. Existing work commonly evaluates a particular scaling policy or applies analytical and simulation-based models to a specific system. Relatively few studies compare distinct FaaS autoscaling strategies under common workload and system assumptions. Moreover, bursty arrivals and practical constraints, including per-instance concurrency, capacity management, and queue buildup near saturation, are not always considered together. A unified framework can therefore support more consistent and reproducible comparisons under dynamic workloads.

3. Background A QN provides a mathematical framework for representing systems in which requests compete for processing Page 3 of 18

Autoscaling in Serverless Computing Table 1 Comparison of Existing Studies and the Proposed Framework Study Tran and Kim [16] Tran and Kim [17] Shafi et al. [18] Zafeiropoulos et al. [19] Khaleq and Ra [20] Benedetti et al. [12] Mampage et al. [21] Li et al. [22] Abdullah et al. [23] Ship et al. [24] Zafarzade and Ataie [25] Wu et al. [26] Fé et al. [27] Gias et al. [28] Luan and Shen [29] Gao et al. [30] Lim et al. [31] Ataie et al. [32] Merkouche et al. [33] Jawaddi et al. [34] Proposed Framework

Analytical/ Formal Modeling

SimulationBased Evaluation

Cross-Strategy Comparative Modeling

Serverless Environment

Bursty Workload

Explicit Request Routing/ Dispatching

RequestConcurrency Constraint

Horizontal Scaling

Vertical Scaling

✗ ✗ ✗ ✗ ✗ ✗ ✓ ✓ ✗ ✓ ✓ ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✗ ✗ ✓ ✓ ✗ ✗ ✓ ✗ ✗ ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✗ ✓

✗ ✗ ✓ ✓ ✗ ✗ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✗ ✗ ✗ ✓ ✗ ✗ ✓ ✓

✓ ✓ ✗ ✓ ✗ ✓ ✓ ✗ ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✓

✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✗ ✗ ✗ ✓ ✗ ✗ ✓ ✗ ✗ ✗ ✓

✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✓ ✗ ✗ ✓

✗ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✓ ✗ ✗ ✓ ✓

✓ ✓ ✗ ✗ ✗ ✗ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✓

resources. A QN consists of interconnected service stations through which requests may arrive, receive service, wait when processing capacity is occupied, and move between stations according to the system structure. Each station can be characterized by parameters such as the arrival process, service-time distribution, number of servers, and scheduling policy. A QN enables performance measures such as throughput, response time, utilization, and system population to be analyzed under different workload and capacity conditions. In this study, a single-station 𝑀∕𝑀∕∞ queueing model is used as the analytical baseline for SPR, representing an idealized system in which each incoming request can immediately obtain processing capacity without queueing [35, 36]. A PN provides a state-based formalism for representing the dynamic behavior of systems with concurrent, synchronized, and conditional activities. A PN consists of places, transitions, tokens, and directed arcs connecting places and transitions. Places represent system conditions or resource states, while tokens located in places represent system entities or quantities associated with these states, such as requests, available capacity, or concurrency permits. The distribution of tokens across places, referred to as the marking, defines the current state of the modeled system. Transitions represent events whose firing changes the system state by consuming tokens from input places and producing tokens in output places when enabling conditions are satisfied. This structure makes PN suitable for representing concurrency, synchronization, resource sharing, and conditional behavior in dynamic systems [37]. QPN combines the state-based representation of PN with queueing and service behavior [38]. In particular, queueing places allow tokens to wait for and receive service according

Author et al.: Preprint submitted to Elsevier

to a specified scheduling strategy and service-time distribution. QPN can therefore represent both the logical evolution of the system state and performance-related effects such as queueing and processing delays. In this study, these formalisms provide the conceptual and analytical basis for modeling the considered autoscaling mechanisms. QN is used to establish the analytical baseline, while PN concepts provide the state-based mechanisms required to represent conditional activation and resource-state changes. QPN incorporates these mechanisms with queueing and service dynamics, providing the basis for modeling the dynamic behavior of the autoscaling strategies developed in Section 5.

4. System Description Autoscaling enables computing capacity to be adjusted dynamically according to workload conditions, reducing the need for direct intervention in resource provisioning. Autoscaling mechanisms can be broadly categorized as horizontal or vertical. Horizontal scaling, also referred to as scale-out/scale-in, adjusts capacity by increasing or decreasing the number of active FIs. In contrast, vertical scaling, or scale-up/scale-down, modifies the computational resources, such as CPU or memory, allocated to an existing FI. The autoscaling strategies evaluated in this research primarily follow the horizontal scaling paradigm, since capacity adaptation is achieved through the activation or deactivation of FIs. Three representative approaches are considered here: SPR, CLS, and MBS. The descriptions in this section provide a conceptual view of these autoscaling approaches, while their corresponding QPN implementations and evaluation settings are presented in Section 5.

Page 4 of 18

Autoscaling in Serverless Computing

(a) SPR

(b) CLS

(c) MBS

Figure 1: Conceptual illustrations of the autoscaling approaches considered in this study: (a) SPR, (b) CLS, and (c) MBS.

SPR. In SPR [39], processing capacity is adapted on a perrequest basis. As illustrated in Fig. 1(a), incoming requests are directed to a request router/scheduler, which assigns each request to an available FI. Each FI can process at most one request at a time. Therefore, if an idle FI is available, it can be reused to process a new request. If no idle FI is available when a request arrives, an additional FI is activated to provide the required processing capacity. After completing the assigned task, the FI is retained in a warm state for a predefined idle period and becomes available for subsequent requests during this period [40]. If no new request arrives within the defined retention interval, the FI is released and its allocated resources are removed from the active capacity pool. Thus, in SPR, processing capacity is adapted directly according to request arrivals and FI availability, rather than being triggered by a concurrency limit or a monitored system metric [39, 41].

CLS. In CLS [42], each FI is configured with a predefined concurrency capacity, denoted by 𝑐, which determines the maximum number of requests that can be processed simultaneously [43]. As illustrated in Fig. 1(b), incoming requests are directed to a request router/scheduler and assigned to an FI with available concurrency capacity. Requests continue to be assigned to an FI until its concurrency capacity is fully utilized. When the available capacity of all active FIs is exhausted and additional requests require service, another FI is activated, contributing an additional concurrency capacity of 𝑐 [44]. This process continues progressively as the workload increases. When a request completes execution, the corresponding concurrency capacity of the host FI becomes available again and can be reused by newly arriving or waiting requests before additional FIs are activated. After becoming idle, an FI remains available for a predefined retention period. If no new requests are assigned to the FI during this period, the FI is released and the number Author et al.: Preprint submitted to Elsevier

of active FIs is reduced. Thus, CLS performs scale-out when the available concurrency capacity is exhausted and scale-in when excess FIs remain idle beyond the retention period, while reusing available capacity in active FIs whenever possible [42, 44].

MBS. In MBS [45, 46], scaling decisions are based on monitored system metrics that reflect the current workload conditions. Several metrics can be considered for this purpose, including response time, utilization, queue length, throughput, and cost-related indicators. In this study, utilization is used as the primary scaling metric. As illustrated in Fig. 1(c), incoming requests are distributed among the active FIs, while their utilization is monitored. The observed utilization is provided to the autoscaler, which compares it with predefined scaling thresholds [47]. When utilization exceeds the upper threshold, the autoscaler activates an additional FI to increase the available processing capacity. Conversely, when utilization falls below the lower threshold, excess FIs can be deactivated as the workload decreases. The conceptual view in Fig. 1(c) also includes a warm state, representing the temporary retention of an idle FI before it is released in practical serverless environments [47]. Overall, MBS adjusts the number of active FIs according to monitored utilization, providing scale-out under increasing demand and scale-in as workload demand decreases. Table 2 summarizes the main characteristics of the scaling approaches considered in this paper.

5. The Proposed Model This section presents the models developed for the SPR, CLS, and MBS autoscaling strategies. The operational behavior described in Section 4 is represented using the modeling formalisms introduced in Section 3. For each strategy, Page 5 of 18

Autoscaling in Serverless Computing Table 2 Comparison of the SPR, CLS, and MBS autoscaling approaches. Feature

SPR

CLS

MBS

Scaling Type Model Structure

Horizontal One concurrent request per FI

Horizontal Utilization-driven FI activation

Modeling Tool Execution Policy

JSIMgraph, MATLAB Independent per-request processing Request arrival Dedicated FI capacity per request

Horizontal, Vertical Shared FIs with concurrency limits JSIMgraph, Python Concurrency-limited shared processing Concurrency limit Shared FI capacity subject to concurrency limits

Scaling Trigger/Metric Allocated Resources

the model components and request-flow mechanism are described according to their corresponding formalism. The performance metrics used for the comparative evaluation are then introduced.

Figure 2: QPN representation of the SPR architecture.

5.1. Analytical and QPN-Based Autoscaling Models The three autoscaling strategies are represented using complementary modeling formalisms tailored to their operational mechanics. In Section 5.1.1, the SPR strategy is modeled using a combination of an analytical QN and a detailed QPN representation to capture explicit request flows and instance-level capacity management. In Section 5.1.2, the CLS strategy is represented using QPN to capture dynamic concurrency-control behavior and the state-dependent activation of FI capacity. Finally, Section 5.1.3 presents the QPN-based framework for the MBS strategy, where metricdriven scaling decisions and request dispatching algorithms are integrated within a central control component.

Author et al.: Preprint submitted to Elsevier

JSIMgraph, Python Threshold-triggered scaling with routing Utilization FI count adapted according to utilization

5.1.1. SPR Model For SPR, an 𝑀∕𝑀∕∞ queue is first used as an analytical baseline. The model represents an idealized processing system in which each arriving request can immediately obtain processing capacity and therefore does not wait because of a shortage of available FIs. To represent the internal request flow and FI-capacity control explicitly, SPR is also modeled using QPN, as shown in Fig. 2. In Fig. 2, the Source generates incoming requests, which are represented by request tokens in the QPN model. These tokens enter the Arriving Requests place, where they wait until an FI branch with available processing capacity can admit them. The model contains a set of parallel FI branches, represented by FI1 , FI2 , FI3 , …, FI𝑛 . Each branch has a corresponding MaxReq-F𝑖 place containing a capacity token that represents the availability of the associated FI𝑖 . Thus, the presence of a token in MaxReq-F𝑖 indicates that the corresponding FI is available to admit a request. For each branch, the corresponding Join FI𝑖 component is modeled as a transition. It has two required inputs: a request token from the Arriving Requests place and a capacity token from MaxReq-F𝑖 . Therefore, Join FI𝑖 becomes enabled only when both a waiting request and available FI capacity are present. When the transition fires, one request token is removed from Arriving Requests and forwarded to the corresponding FI𝑖 queue, while the associated capacity token is removed from MaxReq-F𝑖 . Removing this capacity token prevents the same FI from admitting another request while its current request is being processed. For example, in the first branch, Join FI1 requires a request token from Arriving Requests and the capacity token stored in MaxReq-F1 . Once both tokens are available, the transition fires and the admitted request is forwarded to FI1 , while the capacity token is temporarily removed from MaxReq-F1 . Each FI𝑖 component is modeled as a queueing/service station that represents request execution by the corr.esponding function instance. After service is completed, the request token leaves FI𝑖 and enters the corresponding Complete𝑖 place. This place represents the state in which processing has finished but the completed request has not yet departed from the model

Page 6 of 18

Autoscaling in Serverless Computing

The corresponding Release𝑖 component is modeled as a transition that performs both request departure and capacity restoration. When Release𝑖 fires, the completed request token is forwarded from Complete𝑖 to the common Sink, which represents the departure of completed requests from the system. At the same time, the capacity token associated with FI𝑖 is returned to MaxReq-F𝑖 . The return of this token indicates that the corresponding FI𝑖 is again available to admit and process another request. The same sequence of QPN components is replicated across all FI branches, including FI1 , FI2 , FI3 , and the subsequent branches up to FI𝑛 . Accordingly, each branch contains its corresponding MaxReq-F𝑖 , Join FI𝑖 , FI𝑖 , Complete𝑖 , and Release𝑖 components, where 𝑖 = 1, … , 𝑛. All FI branches receive request tokens from the common Arriving Requests place, while their release transitions forward completed request tokens to the same Sink. Therefore, multiple requests can be processed concurrently through separate FI branches whenever their corresponding capacity tokens are available. In this way, the QPN model represents the SPR mechanism through request tokens, FI-capacity tokens, admission transitions, parallel service stations, completion places, and release transitions.

5.1.2. CLS Model The CLS strategy is represented using QPN to capture concurrency-limited request admission and the statedependent activation of additional FI capacity. The corresponding model is shown in Fig. 3. The QPN representation explicitly describes how concurrency tokens control request admission and how subsequent FI branches become available for processing.

Figure 3: QPN representation of the CLS architecture with concurrency control and conditional FI activation.

The corresponding model is shown in Fig. 3. Requests generated by the Source enter the Arriving Requests place, Author et al.: Preprint submitted to Elsevier

where they wait until the conditions required for admission to an FI are satisfied. For each FI branch, from FI1 to FI𝑛 , a corresponding MaxReq-F place contains a predefined number of concurrency tokens, where each token represents one unit of concurrency capacity of the associated FI. Therefore, the initial number of tokens in MaxReq-F1 , MaxReq-F2 , MaxReq-F3 , . . . , MaxReq-F𝑛 defines the maximum number of requests that FI1 , FI2 , FI3 , . . . , FI𝑛 can process concurrently, respectively. The Join FI1 , Join FI2 , Join FI3 , . . . , Join FI𝑛 components are modeled as transitions that control the admission of requests from the Arriving Requests place to the corresponding FI processing components. Inhibitor arcs impose the additional conditions required for admitting requests to subsequent FI branches. The Join FI1 component is an admission transition. It becomes enabled when at least one request token is waiting in Arriving Requests and at least one concurrency token is available in MaxReq-F1 . When the transition fires, one request token is admitted to the FI1 service station and one concurrency token is removed from MaxReq-F1 . The consumed concurrency token remains unavailable while the corresponding request is being processed. The FI1 component is modeled as a queueing/service station. Multiple requests can therefore be admitted to this station as long as concurrency tokens remain in MaxReq-F1 . Once all tokens in MaxReq-F1 are consumed, FI1 reaches its configured concurrency limit and cannot admit additional requests until some of its concurrency capacity becomes available again. The admission transitions of subsequent FI branches are additionally controlled by inhibitor arcs. In particular, Join FI2 is inhibited while concurrency capacity remains available in MaxReq-F1 . When MaxReq-F1 becomes empty, the concurrency capacity of FI1 is fully utilized. Under this condition, if another request is waiting in Arriving Requests and a concurrency token is available in MaxReq-F2 , Join FI2 can admit the request to FI2 . The same admission logic is extended to the remaining FI branches. Thus, Join FI3 becomes eligible only after the required preceding FI capacities have been exhausted. The same structure continues through the intermediate FI branches, including Join FI𝑛−1 , up to Join FI𝑛 . In this way, the inhibitor-arc structure enforces sequential utilization of the available FI concurrency capacity. After service at any FI branch is completed, the corresponding request token moves from the FI service station of that branch to its associated Complete place, such as Complete1 , Complete2 , Complete3 , . . . , Complete𝑛 . The associated Release transition of that branch, such as Release1 , Release2 , Release3 , . . . , Release𝑛 , then forwards the completed request token to the common Sink, which represents the departure of completed requests from the system. At the same time, the consumed concurrency token is returned to the corresponding MaxReq-F place of that branch. The returned token indicates that one unit of concurrency capacity has become available again in the corresponding FI.

Page 7 of 18

Autoscaling in Serverless Computing

The return of concurrency tokens also changes the inhibitor conditions dynamically. For example, if FI2 has become available for admission because all tokens in MaxReq-F1 were consumed, the return of a token to MaxReq-F1 indicates that FI1 again has available concurrency capacity. The corresponding inhibitor condition therefore prevents further admission to subsequent FI branches while capacity remains available in the preceding branch. Waiting or newly arriving requests are consequently directed again to the earliest FI with available concurrency capacity.

Figure 4: QPN-based representation of the MBS Sarchitecture.

Thus, the CLS model explicitly combines a request waiting place, request tokens, concurrency-capacity places, concurrency tokens, admission transitions, FI service stations, completion places, release transitions, inhibitor arcs, and a common sink. Together, these QPN elements represent the sequential and state-dependent concurrency behavior of CLS.

5.1.3. MBS Model The MBS strategy is represented using a QPN-based architecture in which metric-driven autoscaling and request dispatching are integrated within the Router. The corresponding model is shown in Fig. 4. In Fig. 4, the Source generates incoming requests, which are represented by request tokens in the QPN model. These tokens first enter the Arriving Requests place, where they wait before entering the control and processing part of the model. The Join component is modeled as a transition that admits waiting request tokens from Arriving Requests and forwards them to the Router, where the routing decision is performed. The Router acts as the central control component of the MBS model and performs both metric-driven autoscaling and request dispatching. During system operation, the Router evaluates the selected scaling metric of the active FI pool, such as utilization. The observed metric value is compared with predefined upper and lower thresholds to determine whether the currently available processing capacity should be increased, maintained, or reduced.

Author et al.: Preprint submitted to Elsevier

When the monitored metric reaches or exceeds the upper threshold, the Router performs a scale-out operation by making an additional FI available for request processing. Conversely, when the monitored metric reaches or falls below the lower threshold, the Router initiates the scale-in procedure. An FI that becomes idle is not removed immediately. Instead, it remains available for a predefined observation period and is removed only if no new request is assigned to it during that period. Therefore, the number of active FIs is dynamically adjusted according to the monitored workload conditions. The Router also dispatches each admitted request token among the currently active FI branches according to the configured routing policy. Consequently, autoscaling and request dispatching are integrated within the same control component: the autoscaling logic determines the processing capacity currently available, while the routing logic determines how incoming requests are distributed over that capacity. Three request-dispatching algorithms are implemented and evaluated within the Router: random, round-robin, and probabilistic routing. Under random routing, one of the currently active FIs is selected randomly for each request. Under round-robin routing, successive request tokens are distributed cyclically among the active FI branches. Under probabilistic routing, requests are dispatched according to predefined routing probabilities. Thus, the selected routing policy determines how the incoming workload is distributed among the FIs that are active at the time of the routing decision. The FI1 , FI2 , . . . , FI𝑛 components represent the processing elements of the model. Each request token leaving the Router is forwarded to one of the active FI service stations according to the selected dispatching algorithm. As the autoscaling condition changes, the set of active FIs considered by the Router for subsequent request dispatching is updated accordingly. After processing is completed at an FI, the corresponding request token moves to its associated completion place. These places are represented by Complete1 , Complete2 , . . . , Complete𝑛 . The corresponding Release1 , Release2 , . . . , Release𝑛 transitions then forward the completed request tokens to the common Sink, which represents their departure from the system. Consequently, the MBS model consists of a sequence of explicit QPN and control components. The Source generates request tokens, the Arriving Requests place stores waiting requests, and the Join transition admits them to the Router. The Router evaluates the selected scaling metric, performs scale-out or scale-in when the corresponding threshold condition is satisfied, and dispatches admitted requests among the currently active FI branches. The FI1 through FI𝑛 service stations perform request processing, their corresponding Complete places represent completed processing, the associated Release transitions complete the request path, and the common Sink removes completed request tokens from the system.

Page 8 of 18

Autoscaling in Serverless Computing

This structure allows the autoscaling mechanism and the request-dispatching algorithm to be investigated jointly within the same MBS model. The monitored metric and its scaling thresholds determine the number of active FIs, whereas the selected routing algorithm determines how incoming requests are distributed among those active FIs.

5.2. Evaluation Metrics To rigorously evaluate and compare the performance of the considered autoscaling strategies, four system-level performance measures are defined: throughput, mean system response time, in-system population, and system power. These metrics provide a comprehensive perspective on dynamic capacity adaptation and capture the fundamental trade-offs between system responsiveness, usage level, and overall stability under varying workload conditions. The formal mathematical definitions of these metrics are detailed below.

Throughput (𝑋𝑠𝑦𝑠 ) represents the average rate at which requests are successfully processed and departed from the system per time unit. The overall system throughput is defined as: 𝑋𝑠𝑦𝑠 = 𝜆𝑖𝑛

(1)

where 𝜆𝑖𝑛 denotes the aggregate mean arrival rate of incoming requests to the system.

Mean system response time (𝐸[𝑇𝑅𝑠𝑦𝑠 ]) is the average of the total residence time experienced by each request across all visited processing stations. Therefore, it can be computed as the sum of the station-level mean response times (𝐸[𝑇𝑅𝑖 ]), which account for both queueing delay (𝐸[𝑇𝑄𝑖 ]) and service time (𝐸[𝑆 𝑖 ]) at station 𝑖: 𝑛 𝑛 ( ) ∑ ∑ 𝐸[𝑇𝑄𝑖 ] + 𝐸[𝑆 𝑖 ] (2) 𝐸[𝑇𝑅𝑖 ] = 𝐸[𝑇𝑅𝑠𝑦𝑠 ] = 𝑖=1

𝑖=1

where 𝑛 represents the total number of queueing stations in the system.

Utilization (𝑈 𝑖 ) measures the proportion of time a station’s servers are busy, which mathematically corresponds to the mean number of requests currently being served at station 𝑖. For a station with 𝑀𝑖 servers, effective arrival rate 𝜆𝑖 , and mean service time 𝐸[𝑆 𝑖 ], utilization is calculated as: 𝑈𝑖 =

𝜆𝑖 ⋅ 𝐸[𝑆 𝑖 ]

(3)

𝑀𝑖

𝑠𝑦𝑠 In-system population (𝐸[𝑁𝑟𝑒𝑞 ]) quantifies the average

of the total number of requests presented across the entire system. At the station level, the expected population 𝑖 ]) is the sum of the requests waiting in the queue (𝐸[𝑁𝑟𝑒𝑞 (𝐸[𝑁𝑄𝑖 ]) of station 𝑖, and those actively being served (𝑈 𝑖 ) at that station. Therefore, the expected in-system population is expressed as: 𝑛 𝑛 ( ) ∑ ∑ 𝑠𝑦𝑠 𝑖 𝐸[𝑁𝑟𝑒𝑞 ]= 𝐸[𝑁𝑟𝑒𝑞 ]= 𝐸[𝑁𝑄𝑖 ] + 𝑈 𝑖 (4) 𝑖=1

System power (𝑃𝑠𝑦𝑠 ) characterizes the optimal operational efficiency of the system by evaluating the speed–throughput trade-off. It is defined as the ratio of overall system throughput to mean system response time: 𝑃𝑠𝑦𝑠 =

𝑋𝑠𝑦𝑠 𝐸[𝑇𝑅𝑠𝑦𝑠 ]

(5)

A higher system power value indicates an optimal operating state where the system maximizes throughput while keeping latency low.

6. Performance Evaluation Results This section presents the performance evaluation of SPR, CLS, and MBS. Section 6.1 first defines the common experimental settings used throughout the simulation-based evaluations. The results for SPR, CLS, and MBS are then presented, followed by a comparative evaluation of the three strategies.

6.1. Common Experimental Setup Unless otherwise stated, the simulation-based evaluations use the same 𝐻2 request-arrival process. The workload is characterized by a mixing probability of 𝑝 = 0.2 and a squared coefficient of variation of the inter-arrival time of 𝐶𝑎2 ≈ 2.125, corresponding to 𝐶𝑎 ≈ 1.457. This level of variability reflects the bursty nature of the workload. In the JMT What-if analysis, 𝜆 denotes the mean arrival rate of the overall 𝐻2 workload and is used as the workloadcontrol parameter. When 𝜆 is varied, JMT adjusts the mean inter-arrival time while preserving the mixing probability 𝑝 and the coefficient of variation of the arrival process; the underlying phase rates are therefore recalculated accordingly [48, 49, 50]. Accordingly, the workload levels reported in the following experiments represent mean arrival rates rather than constant instantaneous ones. Because the 𝐻2 arrival process generates inter-arrival times probabilistically from its two phases, requests may occasionally arrive in closely spaced groups, creating short-term bursts even when the mean arrival rate remains unchanged.

6.2. SPR Performance Evaluation The SPR strategy is evaluated from both idealized and practical perspectives. First, an unlimited-capacity analytical baseline is validated against simulation results. The analysis is then extended to a capacity-limited implementation to examine performance degradation near saturation and the effects of finite resource availability.

6.2.1. Analytical Baseline and Model Validation To establish an ideal analytical baseline for the SPR strategy, an 𝑀∕𝑀∕∞ model is considered, in which requests arrive according to a Poisson process with rate 𝜆𝑖𝑛 , service times are exponentially distributed with rate 𝜇, and each arriving request can immediately obtain processing

𝑖=1

Author et al.: Preprint submitted to Elsevier

Page 9 of 18

Autoscaling in Serverless Computing

capacity. Under this idealized assumption, no queueing delay is introduced. The analytical calculations are carried out in MATLAB, and the resulting values are compared with unlimitedcapacity simulation results to validate the model implementation. For each arrival rate, five independent simulation runs are performed, and the resulting performance measures are averaged. For the unlimited-capacity model, every arriving request immediately enters service; therefore, system throughput follows the input rate, i.e., 𝑋𝑠𝑦𝑠 = 𝜆𝑖𝑛 , as defined in Equation 1. Since the queueing delay is zero, the mean system response time is determined only by the mean service time. With the service rate set to 𝜇 = 3 req/s, 𝐸[𝑇𝑅𝑠𝑦𝑠 ] = 𝐸[𝑆] =

1 ≈ 0.33 seconds. 𝜇

(6)

The corresponding mean system population is obtained from Little’s Law as 𝑠𝑦𝑠 ] = 𝜆𝑖𝑛 𝐸[𝑇𝑅𝑠𝑦𝑠 ]. 𝐸[𝑁𝑟𝑒𝑞

(7)

The simulation results closely reproduce these analytical relationships. System throughput follows the input arrival rate over the evaluated workload range, as expected from the 𝑀∕𝑀∕∞ formulation, with a maximum relative deviation of approximately 0.3% between the simulated and analytical values. The simulated response time remains close to the analytical value of 0.33 seconds; relatively larger deviations at low arrival rates result from the smaller number of completed requests available for estimating the mean response time. The relative response-time error is approximately 0.49%. The simulated system population also follows the linear behavior predicted by Little’s Law, with an analytical mean population of approximately 5 requests at 𝜆𝑖𝑛 = 15 req/s. Using the throughput and response-time results above in Equation 5, the analytical system power increases linearly with the arrival rate and reaches approximately 45 at 𝜆𝑖𝑛 = 15 req/s. The simulation-derived system power follows the same approximately linear increasing trend as the analytical baseline. This initial validation considers system throughput, mean system response time, system population, and system power. Overall, the agreement between the analytical and simulation results validates the unlimited-capacity SPR implementation and establishes the ideal analytical baseline used for comparison with the capacity-limited SPR model in Section 6.2.2.

6.2.2. Capacity-Limited SPR Performance To evaluate SPR under finite processing capacity, the practical QPN model described in Section 5.1.1 is implemented and evaluated in JMT. Unlike the unlimited-capacity baseline considered in Section 6.2.1, this model is limited to 15 simultaneously active FIs, with each FI processing Author et al.: Preprint submitted to Elsevier

at most one request at a time. The capacity-limited model uses the common 𝐻2 workload defined in Section 6.1. Service times are exponentially distributed with a service rate of 𝜇 = 3 req/s. The simulation is executed with up to 500,000 samples and repeated 10 times, and the reported performance measures are obtained by averaging the corresponding independent runs. As shown in Fig. 5 (a), system throughput increases almost linearly with the mean arrival rate 𝜆 and remains close to the ideal analytical baseline throughout most of the evaluated workload range. The mean relative error between the capacity-limited QPN results and the analytical SPR baseline is approximately 0.54%. The impact of finite capacity becomes considerably more evident in the mean system response time, as illustrated in Fig. 5 (b). Under low and moderate loads, the response time remains relatively stable, although it is approximately 40–50% higher than the ideal unlimited-capacity baseline. As the arrival rate approaches the capacity provided by the 15 parallel FIs, the responsetime curve begins to deteriorate rapidly. Since 𝜆 denotes the mean arrival rate of the bursty 𝐻2 workload rather than a constant instantaneous rate, short-term arrival bursts may occur around a given mean workload level and contribute to the observed increase in delay. A practical breaking point is observed at approximately 𝜆 = 14.5 req/s, beyond which the mean response time increases sharply and reaches approximately 62.37 seconds at 𝜆 = 15 req/s. To further examine the source of the rapid response-time increase, the mean number of requests accumulated in the Arriving Requests input place is also examined (Fig. 5 (c)). This quantity represents the requests that remain in the input place before they can proceed to the processing part of the SPR model. At low and moderate arrival rates, the population of this place remains relatively small. As the workload approaches the high-load region, however, requests begin to accumulate rapidly in the input place. At 𝜆 = 15 req/s, the mean population of the Arriving Requests place reaches approximately 347.67 requests. The simultaneous increase in this population and in the mean system response time indicates that request accumulation at the input of the model becomes a major source of delay near saturation. The same behavior is reflected in the system population, shown in Fig. 5 (d). The number of requests present in the system remains comparatively small throughout the stable operating region but grows rapidly as the workload approaches the capacity boundary. At 𝜆 = 15 req/s, the capacity-limited model contains an average of approximately 416.28 requests. In contrast, the unlimited SPR baseline predicts only about 5 requests at the same arrival rate. This difference shows that the major degradation near the capacity boundary is associated with request accumulation and the resulting queueing delay, even while system throughput remains close to the incoming rate. System power provides a complementary view of this performance transition. As shown in Fig. 5 (e), 𝑃𝑠𝑦𝑠 initially increases as the workload grows because throughput Page 10 of 18

Autoscaling in Serverless Computing

(a) System throughput.

(b) Mean system response time.

(d) System population.

(c) Mean number of requests in the Arriving Requests input place.

(e) System power.

Figure 5: Performance of the capacity-limited SPR model under increasing workload: (a) system throughput, (b) mean system response time, (c) mean number of requests in the Arriving Requests input place, (d) system population, and (e) system power.

increases while response time remains relatively low. The metric reaches a maximum value of approximately 21.1 at 𝜆 = 12.5 req/s. Beyond this point, the increase in response time outweighs the corresponding throughput gain, causing system power to decline. Hence, the power curve indicates that the throughput–latency trade-off begins to deteriorate before the most severe queue accumulation is observed near the capacity limit. Overall, Fig. 5 shows that the capacity-limited SPR model remains close to the ideal baseline at low and moderate workloads, but its behavior changes substantially in the high-load region. Throughput remains comparatively stable, whereas response time, the population of the Arriving Requests place, and total system population increase rapidly as the available FI capacity is approached. These results reveal a clear QoS degradation region that is not captured by the ideal 𝑀∕𝑀∕∞ baseline. Although SPR conceptually provides processing capacity on a per-request basis, practical serverless environments operate under finite resource availability and provider-specific scaling constraints. The 15-FI limit used in the QPN model is not intended to reproduce the configuration of a specific commercial platform; rather, it provides a controlled finite-capacity boundary for examining the departure from the ideal 𝑀∕𝑀∕∞ behavior. Thus, the unlimited model serves as an ideal analytical reference, whereas the capacity-limited QPN model captures the performance degradation that may arise when available processing capacity becomes constrained.

Author et al.: Preprint submitted to Elsevier

6.3. CLS Performance Evaluation The CLS strategy is first evaluated in terms of the dynamic state-dependent activation of FIs and the corresponding response-time behavior. The analysis then investigates the effect of distributing a fixed total nominal concurrency capacity across different server configurations. The simulations are performed with up to 300,000 samples and 20 repetitions for each evaluated configuration, and the reported steady-state performance measures are obtained by averaging the corresponding simulation runs.

6.3.1. Dynamic QPN Performance Evaluation To examine the dynamic behavior of the CLS strategy, the experiment considers three FIs (𝐹 𝐼1 –𝐹 𝐼3 ), each with a nominal concurrency capacity of 15 requests. The activation of 𝐹 𝐼2 and 𝐹 𝐼3 follows the state-dependent mechanism described in Section 5.1, whereby additional processing capacity is made available as the currently active capacity becomes insufficient. For this dynamic experiment, the reference 𝐻2 parameterization defined in Section 6.1 is used, corresponding to a mean arrival rate of 3.75 req/s. A 200second simulation is conducted under this workload. The resulting dynamic behavior is summarized in Fig. 6. As shown in Fig. 6 (a), 𝐹 𝐼1 initially handles most of the incoming workload and repeatedly operates close to its concurrency limit. As its available capacity decreases, subsequent requests are admitted to 𝐹 𝐼2 . During periods of higher demand, particularly around 𝑡 = 100–145 seconds, the concurrent load on 𝐹 𝐼2 increases substantially and 𝐹 𝐼3 is also activated. This behavior illustrates the state-dependent expansion of concurrency capacity when the currently active FIs are no longer sufficient to absorb the incoming workload. Page 11 of 18

Autoscaling in Serverless Computing

(a) Dynamic evolution of concurrent executions under CLS scaling.

(b) Response-time evolution of FIs under CLS scaling.

Figure 6: Dynamic performance of the CLS model under the 𝐻2 workload: (a) concurrent executions over time and (b) responsetime evolution of the FIs.

The corresponding response-time behavior is presented in Fig. 6 (b). The response time of 𝐹 𝐼1 varies as its concurrency level changes and exhibits several peaks during periods of increased load. When 𝐹 𝐼2 begins to absorb additional requests, its response time initially remains relatively low but increases as its own concurrency capacity becomes more heavily utilized. The most pronounced increase occurs during the high-load interval around 𝑡 = 125–145 seconds, where the response time of 𝐹 𝐼2 reaches approximately 11 seconds. During the same interval, 𝐹 𝐼3 becomes active and provides additional capacity, although its response time also increases as additional requests are admitted to it. Together, these results show that the QPN model captures the time-dependent use of concurrency capacity as the workload varies. When the capacity of an active FI is exhausted, subsequent requests can be admitted to the next available FI, thereby increasing the available processing capacity. Sustained workload bursts can, however, progressively occupy the concurrency capacity of multiple FIs and lead to increased response time.

6.3.2. Wide vs. Deep Concurrency Configurations To examine the effect of concurrency organization on CLS performance, four configurations (M1–M4) are evaluated while keeping the total concurrency-token budget fixed at 12 tokens. Table 3 CLS configurations with a fixed total concurrency-token budget of 12 tokens. Model

FIs

Tokens per FI

Total tokens

M1 M2 M3 M4

2 3 4 6

6 4 3 2

12 12 12 12

As summarized in Table 3, the configurations range from a deep structure, with fewer FIs and more concurrency tokens per FI, to wider structures, with more FIs and fewer tokens per FI. Thus, the quantity held constant across the four configurations is the total number of concurrency tokens, while the number of FIs and the distribution of those Author et al.: Preprint submitted to Elsevier

tokens among the FIs vary. The service-time characteristics of each FI remain unchanged across all configurations. The comparative results are presented in Fig. 7. The system-population and response-time results in Fig. 7(a) and Fig. 7(b) show that the deep configurations experience QoS degradation at considerably lower arrival rates. M1 begins to accumulate requests rapidly from around 𝜆 = 6, whereas the onset of substantial accumulation is progressively delayed in M2 and M3. M4 exhibits the most controlled behavior over the evaluated range. At 𝜆 = 12 req/s, the system populations are approximately 72,360, 30,880, 8,110, and 145 requests for M1–M4, respectively. The response-time curves follow the same general pattern, indicating that a wider distribution of concurrency capacity delays the onset of severe queueing. The throughput results in Fig. 7(c) provide a consistent view of this behavior. M1 reaches an effective throughput ceiling of approximately 6 req/s, while M2 and M3 sustain progressively higher arrival rates before their throughput begins to level off. In contrast, M4 follows the offered workload over almost the entire evaluated range. The observed behavior can be explained by the number of FIs available for request processing: The service rate of each FI is 3 req/s. With 2, 3, 4, and 6 FIs, M1–M4 have aggregate nominal processing rates of approximately 6, 9, 12, and 18 req/s, respectively. As the offered load approaches or exceeds the available processing capacity, requests accumulate, response time increases, and throughput eventually saturates. The higher capacities of wider configurations postpone these effects to higher arrival rates. Because the workload is bursty, temporary request accumulation may occur before these nominal processing limits are reached. System power, shown in Fig. 7(d), exhibits a corresponding shift in its peak as the number of FIs increases. M1 reaches a peak value of approximately 5 around 𝜆 = 5, followed by M2 with a peak of about 9.86 at 𝜆 = 7.5 and M3 with approximately 14.9 at 𝜆 = 9. M4 achieves the highest peak, approximately 19.9 at 𝜆 = 10, and maintains favorable throughput–response-time behavior over a wider workload range before the metric declines. Overall, deeper configurations concentrate the same token budget on fewer processing stations, while wider configurations distribute it across more parallel FIs. M4 therefore exhibits the most Page 12 of 18

Autoscaling in Serverless Computing 25000

M1 M2 M3 M4

70000 60000

M1 M2 M3 M4

20000

50000

E[TRsys]

sys] E[Nreq

15000

40000 30000

10000

20000 5000

10000 0

0

0

1

2

3

4

5

6

7

8

9

10

11

12

0

1

2

(a) System population. 12

3

4

5

6

7

8

9

10

11

12

10

11

12

(b) Mean system response time.

M1 M2 M3 M4

20.0 17.5

M1 M2 M3 M4

10

15.0 12.5

E[Psys]

Xsys

8

6

10.0 7.5

4

5.0 2

2.5 0.0

0

0

1

2

3

4

5

6

7

8

9

10

11

12

(c) System throughput.

0

1

2

3

4

5

6

7

8

9

(d) System power.

Figure 7: Performance comparison of the CLS configurations with a fixed total concurrency-token budget: (a) system population, (b) mean system response time, (c) system throughput, and (d) system power.

favorable performance among the evaluated CLS configurations.

6.4. MBS Performance Evaluation Section 6.4.1 first examines the sensitivity of MBS to different utilization thresholds and their effect on scaling behavior. Section 6.4.2 then investigates the influence of different routing policies on request distribution and system performance. The resulting configuration is subsequently used in Section 6.5 for comparison with the SPR and CLS strategies. The JMT-based MBS simulations are performed with up to 1,000,000 samples and five repetitions for each evaluated setting. The reported performance measures are obtained by averaging the corresponding simulation runs.

6.4.1. Utilization Threshold Sensitivity The utilization threshold determines when MBS makes additional FIs available as the workload increases. To examine its effect on scaling behavior, three threshold values, 𝑈th ∈ {0.7, 0.8, 0.9}, are evaluated under the same workload conditions. Fig. 8(a) compares the utilization profiles obtained for the three settings. A lower threshold causes new FIs to become active earlier, whereas increasing the threshold allows the currently active FIs to operate at higher utilization levels before additional capacity is introduced. Author et al.: Preprint submitted to Elsevier

This shift in activation behavior is more clearly visible in Fig. 8(b). With 𝑈th = 0.7, capacity is expanded earlier as the arrival rate increases, leaving greater utilization headroom for subsequent workload bursts. At 𝑈th = 0.8, FI activation is delayed, while the 0.9 setting keeps the active FIs closer to their utilization limit before further capacity is introduced. The per-FI results also show that increasing the threshold progressively shifts the activation of later FIs toward higher arrival rates. These results illustrate the trade-off introduced by the utilization threshold. Lower values provide a larger operating margin against sudden workload increases but require earlier provisioning of additional FIs. Higher values make greater use of the capacity already available, while leaving less headroom when the workload changes abruptly.

6.4.2. Routing Policy Sensitivity Following the threshold analysis, the utilization threshold is fixed at 𝑈th = 0.7 to examine the effect of request routing under the same scaling threshold. Three routing policies are considered: probabilistic, random, and roundrobin. Their performance is evaluated in terms of system throughput, population, response time, and system power, as shown in Fig. 9. The throughput curves in Fig. 9(a) remain close across the three policies over the evaluated workload range. This indicates that changing the routing policy has only a limited Page 13 of 18

Autoscaling in Serverless Computing Threshold 0.7

Threshold 0.8

Threshold 0.7

1.0

Threshold 0.9

Threshold 0.8

Threshold 0.9

0.8

Utilization

0.6

0.4

0.2

0.0 0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

0

1

2

3

4

5

6

Arrival Rate

7

8

9

10

11

12

13

14

15

0

1

2

3

4

5

6

Arrival Rate FI1

FI2

FI3

FI4

7

8

9

10

11

12

13

14

15

Arrival Rate

FI5

FI6

FI7

FI8

(a) Utilization behavior for 𝑈th = 0.7, 0.8, and 0.9. Utilization by Function Instance under Different Thresholds Threshold 0.7

Threshold 0.8

Threshold 0.9

FI1

FI2

FI3

FI4

FI5

FI6

FI7

FI8

1.0

Utilization

0.8

0.6

0.4

0.2

0.0

1.0

Utilization

0.8

0.6

0.4

0.2

0.0 0

1

2

3

4

5

6

7 8 9 Arrival Rate

10

11

12

13

14

15

0

1

2

3

4

5

6

7 8 9 Arrival Rate

10

11

12

13

14

15

0

1

2

3

4

5

6

7 8 9 Arrival Rate

10

11

12

13

14

15

0

1

2

3

4

5

6

7 8 9 Arrival Rate

10

11

12

13

14

15

(b) Per-FI utilization and activation behavior across the evaluated thresholds. Circles, crosses, and stars denote 𝑈th = 0.7, 0.8, and 0.9, respectively. Figure 8: Sensitivity of MBS scaling behavior to the utilization threshold: (a) utilization profiles across the three evaluated thresholds and (b) activation behavior of individual FIs.

effect on the overall request completion rate. Differences become more apparent in the system population shown in Fig. 9(b). As the arrival rate increases, round-robin maintains a lower request population than the random and probabilistic policies, indicating a more balanced distribution of requests among the active FIs. A similar pattern appears in the response-time results in Fig. 9(c). The three policies perform similarly at low arrival rates, but their behavior diverges as the workload approaches the upper end of the evaluated range. At 𝜆 = 15 req/s, the mean response time reaches approximately 23.37 seconds for round-robin, compared with 33.0 seconds for probabilistic routing and 71.1 seconds for random routing. Thus, round-robin limits the growth in delay more effectively under the considered high-load conditions. Fig. 9(d) shows the corresponding system-power behavior. Since system power combines throughput and response time, the differences among the routing policies become more visible as response time begins to increase. The curves reach their highest region at approximately 𝜆 = 10.5 req/s

Author et al.: Preprint submitted to Elsevier

and decline thereafter as the increase in response time becomes more pronounced. The local variations in system population and response time are mainly associated with threshold-based FI activation and request redistribution. Activating another FI temporarily reduces queueing by distributing incoming requests across more active FIs, after which these measures increase again as the workload grows. The bursty 𝐻2 workload can further contribute to these variations, which are also reflected in system power through changes in response time. Among the evaluated routing policies, round-robin provides the most favorable behavior in the higher-load region. Based on these results, round-robin routing with 𝑈th = 0.7 is adopted for the MBS configuration used in the comparative evaluation of Section 6.5.

6.5. Comparative Evaluation of Autoscaling Strategies The SPR, CLS, and MBS strategies are compared under the workload conditions defined in Section 6.1 to examine

Page 14 of 18

Autoscaling in Serverless Computing Probibility Random RoundRobin

14 12

1044.4

Probibility Random RoundRobin

1000 800

sys] E[Nreq

Xsys

10 8 6

600 400 235.5 315.1

4

200

2

0

1

2

3

4

5

6

7

8

(req/sec)

9

10

11

12

13

14

0

15

3.1 3.1 3.1

0.7 0.7 0.7

0.0 0.0 0.0

0

0

1

2.2 2.8 2.4

2

3

(a) System throughput. 71.1

40

33.0

30

E[Psys]

E[TRsys] (sec)

6

7.8 7.8 5.9

10.6 11.2 8.3

12.4 10.7 8.1

14.2 14.3 10.7

15.5 14.7 11.2

14.5 19.0 15.1

23.3 36.5 22.5 36.6 17.9 29.6

8

9

10

11

12

13

14

15

11

12

13

14

15

7

(req/sec)

8

50

6 4

20

0

5

8.1 8.1 6.3

Probibility Random RoundRobin

10

60

10

4

6.2 5.1 4.1

(b) System population.

Probibility Random RoundRobin

70

5.6 5.6 4.7

23.3 0.5 0.5

0.7 0.7

1.5 1.5

0.9 1.0

1.4 1.4

1.2 1.0

1.3 1.3

0

1

2

3

4

5

6

1.1 1.1

1.3 1.4

1.4 1.2

1.4 1.4

1.4 1.3

1.6 1.6

1.8 1.7

7

8

9

10

11

12

13

(req/sec)

2

2.7 2.5 2.3

14

0

15

(c) Mean system response time.

0

1

2

3

4

5

6

7

8

(req/sec)

9

10

(d) System power.

Figure 9: Sensitivity of MBS performance to the routing policy under 𝑈th = 0.7: (a) system throughput, (b) system population, (c) mean system response time, and (d) system power.

how their different scaling mechanisms affect system behavior as the offered load increases. The mean arrival rate is varied from 𝜆 = 0 to 15 req/s over the evaluated workload range. For this investigation, SPR can activate up to 15 FIs, CLS is configured with five FIs and three concurrency tokens per FI, and MBS uses the utilization threshold 𝑈th = 0.7 with round-robin routing. The comparison considers system population, throughput, response time, and system power, as summarized in Fig. 10. Although all simulations cover the complete workload range up to 𝜆 = 15 req/s, the systempopulation and mean-response-time plots are displayed only up to 𝜆 = 12.5 req/s. At higher arrival rates, the corresponding values, particularly for CLS, increase sharply; displaying these points on the same scale would compress the lowerload region and obscure the differences among the strategies. The high-load results are therefore reported explicitly in the following discussion. The system-population results in Fig. 10(a) show similar behavior among the three strategies at low workloads, but their responses increasingly diverge as the arrival rate grows. SPR maintains the lowest population over most of the lowto-moderate workload range (0 ≤ 𝜆 ≤ 12.5 req/s), reflecting its ability to assign processing capacity directly to incoming requests while sufficient FIs remain available. CLS begins to accumulate requests earlier as its fixed concurrency configuration increasingly restricts request admission across the available FIs. This effect becomes particularly pronounced near the upper end of the workload range. MBS exhibits somewhat higher population than SPR under moderate loads because capacity expansion is governed by the utilization Author et al.: Preprint submitted to Elsevier

threshold rather than by each individual arrival; however, when the utilization threshold is reached, additional processing capacity is activated, which limits queue growth more effectively as the system approaches its capacity boundary. Based on the complete simulation results, at 𝜆 = 15 req/s, the system population reaches approximately 416.3 requests for SPR and 15,300 requests for CLS, compared with approximately 315.1 requests for MBS. The substantially larger CLS population at this workload reflects the severe request accumulation occurring beyond the range displayed in Fig. 10(a). A different pattern is observed for system throughput in Fig. 10(b). All three strategies closely follow the offered workload through the low- and moderate-load regions, and their differences remain small until the arrival rate approaches the available processing capacity. SPR sustains a throughput of approximately 14.92 req/s at 𝜆 = 15 req/s, while MBS reaches approximately 14.82 req/s. CLS begins to level off earlier and provides approximately 12.9 req/s at the same arrival rate. Thus, SPR and MBS preserve most of the incoming throughput near the upper workload limit, whereas the fixed CLS configuration reaches its effective processing limit earlier as the workload increases. The response-time results provide a clearer indication of the different QoS behavior of the three strategies. As illustrated in Fig. 10(c), SPR provides the lowest response time through most of the low-to-moderate workload range, while MBS remains relatively close to it. CLS shows a more noticeable increase as its concurrency capacity becomes

Page 15 of 18

Autoscaling in Serverless Computing

(a) System population.

(b) System throughput.

(c) Mean system response time.

(d) System power.

Figure 10: Comparative performance of SPR, CLS, and MBS: (a) system population, (b) system throughput, (c) mean system response time, and (d) system power.

progressively occupied. The differences become substantially larger near the maximum evaluated load. Based on the complete simulation results beyond the displayed range of Fig. 10(c), at 𝜆 = 15 req/s, the mean response time rises to approximately 62.4 seconds for SPR and 3240 seconds for CLS, whereas MBS limits the corresponding value to approximately 23.27 seconds. These results indicate that SPR provides favorable latency while sufficient per-request capacity is available, but its response time deteriorates rapidly when the finite FI boundary is approached. The utilizationdriven activation used by MBS introduces additional capacity before this condition becomes as severe, resulting in a more gradual increase in delay. Finally, Fig. 10(d) compares system power, defined as sys the ratio 𝑃sys = 𝑋sys ∕𝐸[𝑇𝑅 ]. SPR achieves the highest peak value, approximately 21.1, around 𝜆 = 12.5 req/s, reflecting its combination of high throughput and low response time in this operating region. CLS reaches a lower peak of approximately 16.5 around 𝜆 = 10.5–11 req/s and then declines sharply as its response time increases. MBS has a lower peak, approximately 11.19, but exhibits a more gradual change as the workload increases. Consequently, the power results should not be interpreted as showing a single strategy that dominates across all operating conditions. Rather, they reinforce the trade-off observed in the other

Author et al.: Preprint submitted to Elsevier

metrics: SPR provides the strongest performance under lowto-moderate loads, CLS is more sensitive to concurrencycapacity exhaustion, and MBS sacrifices some peak performance in favor of more controlled behavior near the capacity limit.

7. Conclusion and Future Work This study examined the behavior of the three autoscaling strategies under dynamic and bursty workloads. Analytical queueing models and QPN models were used to capture their different scaling mechanisms, while an 𝐻2 arrival process was used to represent workload variability. For SPR, the 𝑀∕𝑀∕∞ model provided an analytical reference for the corresponding simulation model. Under finite processing capacity, SPR maintained low response time over much of the evaluated workload range, but its performance deteriorated rapidly as the available FI capacity was approached. The CLS results showed that performance depends not only on the total concurrency-token budget, but also on how this budget is distributed across FIs. In the wide-versus-deep experiments, distributing the same token budget across a larger number of independent FIs delayed queue buildup and provided more favorable behavior under increasing load. For MBS, the utilization threshold determined the timing of FI activation, while the routing policy affected how requests were distributed among active FIs. Among the evaluated routing policies, round-robin produced the most

Page 16 of 18

Autoscaling in Serverless Computing

controlled behavior. The configuration with 𝑈th = 0.7 was used in the final comparison with SPR and CLS. The comparative results show that the three strategies exhibit different trade-offs. SPR provides strong responsiveness while sufficient FI capacity is available, whereas CLS becomes more sensitive to concurrency exhaustion as the workload increases. MBS provides a more gradual response near the capacity boundary through utilization-driven FI activation. The results also show that the variability introduced by the 𝐻2 workload can lead to queue growth and QoS degradation before nominal capacity is fully reached. Accordingly, the choice of autoscaling strategy depends on the desired balance between responsiveness, concurrency control, available capacity, and robustness to workload variability. Future work can extend the proposed framework to heterogeneous workloads and FIs with different service demands and resource characteristics. Shared external resources, such as databases, storage services, and external APIs, can also be incorporated into the QPN models to capture contention beyond individual FIs. In addition, resource cost can be incorporated to support multi-objective scaling decisions that balance QoS and resource usage, while predictive mechanisms can be investigated to anticipate workload changes before utilization thresholds are reached. The evaluated strategies can also be implemented on a serverless platform such as Knative or OpenFaaS. Comparing platform measurements with the analytical and simulation results would provide an empirical assessment of the proposed models and help identify effects that are not captured by the current abstraction.

Declaration of Generative AI and AI-assisted Technologies in the Writing Process During the preparation of this work the authors used ChatGPT in order to improve language and readability. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the manuscript.

References [1] S. Hong, Y. Kim, J. Nam, S. Kim, On the analysis of inter-relationship between auto-scaling policy and QoS of FaaS workloads, Sensors 24 (12) (2024) 3774. [2] M. Tari, M. Ghobaei-Arani, J. Pouramini, M. Ghorbian, Auto-scaling mechanisms in serverless computing: A comprehensive review, Computer Science Review 53 (2024) 100650. [3] S. Agarwal, et al., Dynamic function configuration and its management in serverless computing, ACM Computing Surveys (2024). [4] C. Fan, et al., Puffer: Reducing cold-start time in serverless computing, IEEE Transactions on Parallel and Distributed Systems (2024). [5] Y. Li, et al., Lightweight and holistic-scalable serverless secure container runtime, IEEE Transactions on Computers (2024). [6] Z. Karamzadeh, D. Shameli-Sendi, Reducing cold start delay in serverless computing using lightweight virtual machines, Journal of Network and Computer Applications (2024). [7] S. Alharthi, A. Alshamsi, A. Alseiari, A. Alwarafy, Auto-scaling techniques in cloud computing: Issues and research directions, Sensors (Basel, Switzerland) 24 (2024). doi:10.3390/s24175551.

Author et al.: Preprint submitted to Elsevier

[8] F. Meng, H. Dai, G. Cong, B. Zhu, H. Zhao, CATScaler: A convolution-augmented transformer scaling framework for cloudnative applications, IEEE Transactions on Services Computing 18 (2025) 2659–2672. doi:10.1109/tsc.2025.3592383. [9] B. Jeong, Y.-S. Jeong, ARAScaler: Adaptive resource autoscaling scheme using ETimeMixer for efficient cloud-native computing, IEEE Transactions on Services Computing 18 (2025) 72–84. doi:10.1109/ tsc.2024.3522815. [10] A. Ebrahimi, M. Ghobaei-Arani, H. Saboohi, Cold start latency mitigation mechanisms in serverless computing: Taxonomy, review, and future directions, J. Syst. Archit. 151 (2024) 103115. doi:10. 1016/j.sysarc.2024.103115. [11] X. Cai, K. Yan, Y. Gong, C. Hu, D. Cheng, SLO-aware instance management with queuing-based delay execution, IEEE Transactions on Services Computing 18 (2025) 4167–4180. doi:10.1109/tsc.2025. 3632416. [12] P. Benedetti, M. Femminella, G. Reali, Management of autoscaling serverless functions in edge computing via Q-learning, Future Generation Computer Systems 175 (2026) 108112. doi:10.1016/j.future. 2025.108112. [13] X. Cai, Q. Sang, C. Hu, Y. Gong, K. Suo, X. Zhou, D. Cheng, Incendio: Priority-based scheduling for alleviating cold start in serverless computing, IEEE Transactions on Computers 73 (2024) 1780–1794. doi:10.1109/tc.2024.3386063. [14] G. Quattrocchi, E. Incerto, R. Pinciroli, C. Trubiani, L. Baresi, Autoscaling solutions for cloud applications under dynamic workloads, IEEE Transactions on Services Computing 17 (2024) 804–820. doi: 10.1109/tsc.2024.3354062. [15] V. Kulkarni, N. Reddy, T. Khare, A. S. Prasad, C. Babu, Y. L. Simmhan, Characterizing FaaS workflows on public clouds: The good, the bad and the ugly, IEEE Transactions on Parallel and Distributed Systems 37 (2025) 1312–1329. doi:10.1109/tpds.2026. 3678606. [16] M.-N. Tran, Y. Kim, Concurrent service auto-scaling for Knative resource quota-based serverless system, Future Generation Computer Systems 160 (2024) 326–339. doi:10.1016/j.future.2024.06.019. [17] M.-N. Tran, Y. Kim, Optimized resource usage with hybrid autoscaling system for Knative serverless edge computing, Future Generation Computer Systems 152 (2024) 304–316. [18] N. Shafi, M. Abdullah, W. Iqbal, F. Bukhari, CEMA: Cost effective multi-layered autoscaling for microservice based applications, Journal of Network and Computer Applications 242 (2025) 104266. doi:10.1016/j.jnca.2025.104266. [19] A. Zafeiropoulos, E. Fotopoulou, N. Filinis, S. Papavassiliou, Reinforcement learning-assisted autoscaling mechanisms for serverless computing platforms, Simulation Modelling Practice and Theory 116 (2022) 102461. doi:10.1016/j.simpat.2021.102461. [20] A. A. Khaleq, I. Ra, Intelligent microservices autoscaling module using reinforcement learning, Cluster Computing 26 (2023) 2789– 2800. doi:10.1007/s10586-023-03999-8. [21] A. Mampage, S. Karunasekera, R. Buyya, A deep reinforcement learning based algorithm for time and cost optimized scaling of serverless applications, Future Generation Computer Systems 173 (2025) 107873. doi:10.1016/j.future.2025.107873. [22] H. Li, W. Rao, B. Hu, Y. Tian, J. Shen, Energy-aware elastic scaling algorithm for microservices in Kubernetes clouds, Journal of Network and Computer Applications 242 (2025) 104218. doi:10.1016/j.jnca. 2025.104218. [23] M. Abdullah, R. Munir, W. Iqbal, S. A. Khan, S. U. R. Baig, A. Erradi, Cost and performance-effective dynamic VM type selection in autoscaling using reinforcement learning, Computing 107 (9) (2025) 177. doi:10.1007/s00607-025-01535-7. [24] H. Ship, E. Shindin, C. Wang, D. Arroyo, A. Tantawi, Optimizing simultaneous autoscaling for serverless cloud computing, in: 2024 IEEE 17th International Conference on Cloud Computing (CLOUD), IEEE, 2024, pp. 105–114. doi:10.1109/CLOUD62652.2024.00022. [25] Z. Zafarzade, E. Ataie, Capacity planning of a microservices-based image classification application using analytic modeling, Computing

Page 17 of 18

Autoscaling in Serverless Computing 107 (2025) 123. doi:10.1007/s00607-025-01473-4. [26] J. Wu, M. Xu, Y. He, K. Ye, C. Xu, CloudNativeSim: A toolkit for modeling and simulation of cloud-native applications, Software: Practice and Experience 55 (2025) 1185–1208. doi:10.1002/spe.3417. [27] I. Fé, R. Matos, J. Dantas, C. Melo, T. A. Nguyen, D. Min, E. Choi, F. A. Silva, P. Maciel, Performance-cost trade-off in auto-scaling mechanisms for cloud computing, Sensors 22 (3) (2022) 1221. doi: 10.3390/s22031221. [28] A. U. Gias, G. Casale, M. Woodside, ATOM: Model-driven autoscaling for microservices, in: 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), IEEE, 2019, pp. 1994– 2004. doi:10.1109/ICDCS.2019.00197. [29] S. Luan, H. Shen, Minimize resource cost for containerized microservices under SLO via ML-enhanced layered queueing network optimization, in: 2024 14th International Conference on Cloud Computing, Data Science & Engineering (Confluence), IEEE, 2024, pp. 631–637. doi:10.1109/Confluence60223.2024.10463310. [30] Y. Gao, G. Casale, R. Singhal, Performance modeling of distributed data processing in microservice applications, ACM Transactions on Modeling and Performance Evaluation of Computing Systems 9 (4) (2024) 1–30. doi:10.1145/3687265. [31] D. Lim, T. A. Nguyen, J. Li, E. Choi, D. Min, R. M. Cadena, B. Nogueira, E. Andrade, G. Callou, M. Di Mauro, M. Torquato, F. A. Silva, P. Maciel, S-iNAS: A performance-centric SRN modeling and analysis of time and event based scaling strategies in Ceph-based industrial network-attached storage, Journal of Network and Systems Management 34 (1) (2026) 34. doi:10.1007/s10922-025-10005-6. [32] E. Ataie, R. Entezari-Maleki, S. E. Etesami, B. Egger, L. Sousa, A. Movaghar, Modeling and evaluation of dispatching policies in IaaS cloud data centers using SANs, Sustainable Computing: Informatics and Systems 33 (2022) 100617. doi:10.1016/j.suscom.2021.100617. [33] S. Merkouche, C. Bouanaka, E. Benkhelifa, A Petri net-based formal modeling for microservices auto-scaling, in: 2023 20th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA), IEEE, 2023, pp. 1–8. doi:10.1109/AICCSA59173.2023. 10479351. [34] S. N. Agos Jawaddi, A. Ismail, M. S. Sulaiman, V. Cardellini, Analyzing energy-efficient and Kubernetes-based autoscaling of microservices using probabilistic model checking, Journal of Grid Computing 23 (1) (2025) 3. doi:10.1007/s10723-024-09789-9. [35] J. R. Jackson, Networks of waiting lines, Operations Research 5 (4) (1957) 518–521. doi:10.1287/opre.5.4.518. [36] J. R. Jackson, Jobshop-like queueing systems, Management Science 10 (1) (1963) 131–142. doi:10.1287/mnsc.10.1.131. [37] T. Murata, Petri nets: Properties, analysis and applications, Proceedings of the IEEE 77 (4) (1989) 541–580. doi:10.1109/5.24143. [38] F. Bause, Queueing Petri nets: A formalism for the combined qualitative and quantitative analysis of systems, in: Proceedings of the 5th International Workshop on Petri Nets and Performance Models, IEEE Computer Society Press, 1993, pp. 14–23. doi:10.1109/PNPM.1993. 393439. [39] N. Mahmoudi, H. Khazaei, Performance modeling of serverless computing platforms, IEEE Transactions on Cloud Computing 10 (4) (2022) 2834–2847. doi:10.1109/TCC.2020.3033373. [40] P. Kumar, R. Sharma, Y. Liu, Mitigating cold start latency and optimizing warm-pool retention in FaaS platforms, ACM Transactions on Computer Systems 43 (1) (2025) 12–29. [41] J. Anselmi, B. Gaujal, L.-S. Rebuffi, Non-stationary gradient descent for optimal auto-scaling in serverless platforms, IEEE/ACM Transactions on Networking 33 (4) (2025) 1574–1587. doi:10.1109/TON. 2025.3538982. [42] Google Cloud, Maximum concurrent requests for services, Google Cloud Run Documentation, accessed: September 16, 2026. URL https://docs.cloud.google.com/run/docs/about-concurrency [43] X. Li, Y. Wang, H. Zhang, Serverless cold start performance optimization based on multi-request processing and adaptive hierarchical scaling, IEEE Access 12 (2024) 136248–136262.

Author et al.: Preprint submitted to Elsevier

[44] Y. Zhao, X. Chen, M. Guo, ICPS: Real-time resource configuration for cloud serverless functions considering invocation concurrency, IEEE Transactions on Cloud Computing 13 (2) (2025) 201–215. [45] Amazon Web Services, Configuring provisioned concurrency for a function, AWS Lambda Developer Guide, accessed: September 16, 2026. URL https://docs.aws.amazon.com/lambda/latest/dg/ provisioned-concurrency.html

[46] N. Mahmoudi, H. Khazaei, Performance modeling of metric-based serverless computing platforms, IEEE Transactions on Cloud Computing 11 (2) (2023) 1899–1910. doi:10.1109/TCC.2022.3169619. [47] X. Liu, M. Zhang, W. Chen, J. Wang, Adaptive function service auto-scaling for serverless computing: A multi-metric approach, IEEE Transactions on Services Computing 19 (2) (2026) 312–327. [48] G. Serazzi, Impact of variability of interarrival and service times, in: Performance Engineering: Learning Through Applications Using JMT, Springer, Cham, 2024, pp. 63–72. doi:10.1007/ 978-3-031-36763-2_4. [49] Java Modelling Tools Project, Java Modelling Tools User Manual, version 1.4.0 Edition (2026). [50] M. Bertoli, G. Casale, G. Serazzi, JMT: Performance engineering tools for system modeling, ACM SIGMETRICS Performance Evaluation Review 36 (4) (2009) 10–15. doi:10.1145/1530873.1530877.

Page 18 of 18

Related documents

• A Unified Queueing-Based Framework for Comparative Evaluation of Serverless Autoscaling Strategies under Bursty Workloads#1459034
• An interpretable universal bound for multiserver queues via a leave-one-out technique#665551
• An interpretable universal bound for multiserver queues via a leave-one-out technique#664051
• Quantum Broker: QoS-Aware Distribution of Quantum Computations#1412311
• Quantum Broker: QoS-Aware Distribution of Quantum Computations#1406547