arXiv:2605.13496v1 [cs.DC] 13 May 2026
MARLIN: Multi-Agent Game-Theoretic Reinforcement Learning for Sustainable LLM Inference in Cloud Datacenters Hayden Moore
Sirui Qi
Dejan Milojicic
[email protected] Colorado State University Fort Collins, CO, USA
[email protected] Colorado State University Fort Collins, CO, USA
[email protected] Hewlett Packard Labs Milpitas, CA, USA
Cullen Bash
Sudeep Pasricha
[email protected] Hewlett Packard Labs Milpitas, CA, USA
[email protected] Colorado State University Fort Collins, CO, USA
Abstract Large Language Models (LLMs) have become increasingly prevalent in cloud-based platforms, propelled by the introduction of AI-based consumer and enterprise services. LLM inference requests in particular account for up to 90% of total LLM lifecycle energy use, dwarfing training energy costs. The rising volume of LLM inference requests is increasing environmental footprints, particularly carbon emissions and water consumption. To improve sustainability for LLM inference serving in cloud datacenter environments, we propose a novel multi-agent game-theoretic reinforcement learning framework called MARLIN to co-optimize time-to-first token (TTFT), carbon emissions, water usage, and energy costs associated with LLM inference. MARLIN demonstrates a reduction of at least 18% in TTFT, 33% in carbon emissions, 43% in water usage, and 11% in energy costs compared to state-of-the-art LLM inference management frameworks.
CCS Concepts • Computing methodologies → Machine learning; • Information systems; • Computer systems organization → Real-time systems;
Keywords Large language models, Sustainability, Carbon emissions, Water usage, Energy costs, Cloud datacenters, Reinforcement learning
1
Introduction
Large Language Models (LLMs) have experienced explosive growth in recent years, with enterprise adoption expected to exceed 80% in 2026 and global generative AI spending projected to reach $644 billion [1]. This growth is reflected in usage data from large AI companies, with OpenAI reporting over 2.5 billion prompts per day from users across its ChatGPT deployments in 2025 [2]. With this rapid growth in LLM adoption, the associated energy usage in cloud datacenters hosting these LLMs is also rapidly increasing, with an estimated 25-33% compound annual growth rate [3]. The rapid increase in LLM-associated energy consumption in cloud datacenters is accelerating the already high datacenter energy footprint. In 2024, U.S. datacenters consumed 183 terawatt-hours (TWh) of electricity, which is 4% of the total electricity generated in the country. By 2030, this figure is projected to grow by 133% to 426 TWh [4]. This increase is expected to be driven by demand for
both LLM training and serving user inference requests. In the past, the training phase was the focus for improving the sustainability of LLMs. However, recent data suggest that inference dominates LLM lifecycle energy usage, accounting for 90% of the overall energy used in this stage [5]. Hence, it is increasingly important to focus on reducing energy costs and improving the sustainability of LLMs by reducing the resources required to serve LLM inference requests. We target two important sustainability metrics: operational carbon emissions and water usage. Carbon emissions have been a long-term concern associated with high energy usage. LLM inference is currently scaling quickly and is already becoming a concern for carbon emissions. As an example, the yearly carbon emissions from the inference serving of just the GPT-4o model already surpass those of 30,000 cars in a year [5]. As more competing LLMs launch and infrastructure continues to scale, emissions will only climb. The high water usage associated with cooling LLM-serving datacenters and during electricity generation is also becoming a concern. Cooling accounts for an estimated 29% of datacenter water usage, with electricity generation contributing 71%, so both factors must be considered [6]. For GPT-4o inference only, its annual water consumption is up to 1,579,680 kL, enough to fill more than 500 Olympic-sized pools [5]. Importantly, this consumption refers to evaporated freshwater permanently removed from local ecosystems, straining the infrastructure of large cities, especially since an estimated 30% of cities with over 1 million people worldwide already are facing water scarcity [7]. Clearly, the current path to scaling LLM inference is unsustainable. There is a critical need for serving LLM inference sustainably without sacrificing performance. To achieve this goal, we propose a novel framework called MARLIN, to optimize carbon emissions, water usage, and energy costs associated with LLM inference while ensuring that performance (time-to-first-token (TTFT)) remains acceptable for each incoming request. The novel contributions of our work are: • We develop a novel multi-agent reinforcement learning framework called MARLIN that utilizes a game-theoretic approach to balance among competing agents. • We formulate a multi-objective LLM inference scheduling problem that accurately models the TTFT, carbon emissions, water usage, and energy costs of LLM serving across geodistributed cloud datacenters.
• We perform comprehensive ablation analysis and comparison studies with several state-of-the-art LLM inference scheduling frameworks to demonstrate the flexibility and promise of the MARLIN framework.
2
Related Works
Many prior efforts have addressed workload scheduling in cloud datacenters. The primary focus of optimizing workload scheduling in these works has been to improve performance and reduce overall energy consumption. For instance, Hogade et al. proposed a gametheoretic framework that incorporates network transfer costs to minimize datacenter energy costs [8]. Chen et al. used ant colony optimization to consolidate VMs, reducing both power consumption and thermal hotspots [9]. Recent efforts have shifted towards optimizing sustainability metrics, particularly carbon emissions and water usage. To address the rising datacenter carbon emissions, Qi et al. developed a dual-objective evolutionary optimizer that improved the carbon emissions and throughput of serverless function execution across geo-distributed datacenters [10]. Beena et al. proposed a greedy scheduling heuristic that worked across many distributed datacenters and used an LSTM model for carbon emission prediction [11]. Qi et al. minimized carbon emissions, wastewater, and costs using a hybrid evolutionary-boosting algorithm [12]. These prior approaches do not address sustainable LLM inference scheduling. Most existing LLM inference schedulers prioritize datacenter performance metrics. For example, Mei et al. used mixed-integer linear programming to maximize data flow across distributed GPUs to maximize LLM inference throughput [13]. Patel et al. divided LLM inference into distinct phases, enabling pipeline parallelism to improve throughput [14]. Yang et al. developed a framework that leveraged edge and cloud servers using constraint satisfaction and upper confidence bounds, thereby improving throughput and costs [15]. Recent work is starting to focus on sustainability in LLM inference scheduling. Moore et al. combined a genetic algorithm and machine learning to generate Pareto-optimal solutions for LLM inference serving [16]. However, the approach lacks scalability and has a slow convergence speed. MARLIN addresses these limitations with a multi-agent gametheoretic RL approach that improves convergence speed, preserves optimization quality, and balances sustainability with TTFT performance.
3
Figure 1: The number of individual LLM requests in each epoch (15 minutes) over two weeks [17] by epoch (15-minute intervals) and plot their frequencies in Fig. 1. The distribution of LLM inference requests across epochs is quite diverse, highlighting the challenge of scheduling them across cloud datacenters. We created a custom workload model that used the observed arrival patterns from the real cloud trace and paired them with execution models for two contemporary LLMs, Llama-7B and Llama-70B. Execution profiles for these two LLMs were available across multiple GPUs [18]. Real-time LLM serving requires meeting strict memory constraints. Memory constraints require that each request 𝑖’s memory footprint 𝑀𝐹𝑖 not exceed a server node 𝑛’s maximum available memory. Requests share the underlying model’s weights when possible, to reduce memory overhead. We account for shared weights for a model 𝜐’s memory footprint 𝑀𝐹𝜐 , and each request’s growing KV cache 𝐾𝑉(𝜐,𝑖,𝜏 ) as a function of the number of tokens 𝜏. KV cache growth is proportional to the model size. The total server node memory usage 𝑀 (𝑛,𝑒 ) computed for all tokens 𝑇 at epoch 𝑒 is: 𝑀𝑛,𝑒 = (
𝜐
𝑀𝐹𝜐 ) + (
𝐼𝑒 ∑︁ 𝑇 ∑︁ 𝑖
𝐾𝑉𝜐,𝑖,𝜏 )
(1)
𝜏
where Υ is the total number of all active LLMs and 𝐼𝑒 is the total number of requests at epoch 𝑒. If this memory usage exceeds the node 𝑛’s maximum available memory 𝑀𝑡𝑜𝑡,𝑛 , it results in a queuing of requests until prior requests release their KV cache. Real-time LLM serving also requires meeting strict latency constraints. As many LLM applications (e.g., chatbots) require real-time interactivity, minimizing latency 𝐿𝐴𝑖 for each request is critical. Latency in this case has three components: LLM weight loading time, network latency, and computation time. Weight loading time 𝐿𝐴𝑙𝑜𝑎𝑑,𝑖 depends on whether the model is present on the GPU during execution. We assume each model is present on the node locally in storage. However, the model cannot be used until it is loaded onto the onboard memory. The worst-case loading time depends on the slowest memory bandwidth 𝐵𝑊𝑛 in moving the model to the onboard memory. Therefore, the model weight loading time can be expressed as: 𝐿𝐴𝑙𝑜𝑎𝑑,𝑖 = 𝑀𝐹𝜐 /𝐵𝑊𝑛 . To reduce loading time for additional requests, a node keeps the model present in onboard memory. The network latency 𝐿𝐴 (𝑛𝑒𝑡,𝑖 ) consists of moving the user request to a datacenter. The transfer latency of the user request to the datacenter is dependent on the distance 𝑑𝑖𝑠𝑡 and transmission media
Modeling Overview
Our framework is designed to map incoming LLM inference requests to server nodes across geo-distributed datacenters to leverage natural geographic and temporal variations. We comprehensively model the LLM inference workload and cloud datacenter characteristics across geographic locations, capturing energy costs, carbon emissions, water usage, and data transmission latency for each request.
3.1
Υ ∑︁
LLM Workload Model
Our workload model uses a real-world trace [17] from a ChatGPT service on Azure cloud datacenters. The dataset covers two weeks’ worth of requests for GPT-3 and GPT-4. We aggregate requests 2
characterized by 𝜆𝑚𝑒𝑑𝑖𝑎 [19], which is the propagation latency per unit of distance (e.g., milliseconds per kilometer). Requests may also need to be transmitted between datacenters via an inter-datacenter network, which depends on hop count 𝑅𝑠𝑜𝑢𝑟𝑐𝑒,𝑑𝑒𝑠𝑡 as well as the latency of processing each hop 𝜎ℎ𝑜𝑝 . The network latency 𝐿𝐴𝑛𝑒𝑡,𝑖 can be expressed as:
energy is equal to about 13% of the IT energy [23]: 𝐸𝑖𝑛𝑓 𝑟𝑎,𝑑,𝑒 = 0.13 × 𝐸𝐼𝑇 ,𝑑,𝑒 . The total datacenter energy usage at a datacenter 𝑑 during an epoch 𝑒 consists of the sum of components:
𝐿𝐴𝑛𝑒𝑡,𝑖 = (𝑑𝑖𝑠𝑡 × 𝜆𝑚𝑒𝑑𝑖𝑎 ) + (𝑅𝑠𝑜𝑢𝑟𝑐𝑒,𝑑𝑒𝑠𝑡 × 𝜎ℎ𝑜𝑝 )
However, the cost of using that energy will vary depending on the location of the datacenter 𝑑 and the time-of-day when the epoch 𝑒 occurs, which determines the time-of-use pricing 𝑇𝑂𝑈𝑑,𝑒 set by the utility provider. Therefore, the total cost of energy usage across all the datacenters for epoch 𝑒 is:
𝐸𝑡𝑜𝑡,𝑑,𝑒 = 𝐸𝐼𝑇 ,𝑑,𝑒 + 𝐸𝑐𝑜𝑜𝑙,𝑑,𝑒 + 𝐸𝑖𝑛𝑓 𝑟𝑎,𝑑,𝑒
(2)
At the scheduled server node 𝑛, computation proceeds until the first output token 𝜏 is generated. We assume profiled execution time 𝐿𝐴 (𝑡𝑜𝑡,𝑒𝑥𝑒𝑐,𝑖 ) scales linearly with output token count 𝑇𝑖 . We can safely make this assumption due to batched LLM inference being memory bound on GPUs [20]. This bounds the throughput giving the execution time a linear relationship. To calculate the overall TTFT, all latency components must be combined, with network latency counted twice in the final calculation to account for both sending and receiving the request. Therefore, the total request TTFT across all requests 𝐼 during an epoch 𝑒 can be estimated as: 𝐿𝐴𝑡𝑜𝑡,𝑒 =
𝐼𝑒 ∑︁
(𝐿𝐴𝑙𝑜𝑎𝑑,𝑖 + 2 × 𝐿𝐴𝑛𝑒𝑡,𝑖 + 𝐿𝐴𝑡𝑜𝑡,𝑒𝑥𝑒𝑐,𝑖 /𝑇𝑖 )
𝐶𝑜𝑠𝑡𝑡𝑜𝑡,𝑒 =
3.4
(3)
Datacenter Model
Water Usage Model
𝐺𝑡𝑜𝑡,𝑒 =
Energy Cost Model
𝐷 ∑︁
(𝐺 𝐸,𝑑,𝑒 + 𝐺𝑏𝑙𝑜𝑤,𝑑,𝑒 + 𝐺𝑔𝑟𝑖𝑑,𝑑,𝑒 )
(8)
𝑑
Each server node 𝑛 can operate in preset performance states, each of which dissipates a fraction 𝑃𝐶𝑝𝑠𝑡𝑎𝑡𝑒 of the node’s total thermal design power 𝑇 𝐷𝑃𝑛 . The information technology (IT) node energy over an epoch 𝑒 of duration 𝑡𝑒 is computed as: 𝐸𝐼𝑇 ,𝑛,𝑒 = 𝑃𝐶𝑝𝑠𝑡𝑎𝑡𝑒 × 𝑇 𝐷𝑃𝑛 × 𝑡𝑒
(7)
Water is used in two ways in a datacenter: for direct cooling and for energy generation. The two ways can be further divided into three sub-components, which are location-specific: evaporative 𝐺 𝐸,𝑑,𝑒 , blowdown 𝐺𝑏𝑙𝑜𝑤,𝑑,𝑒 , and grid-based 𝐺𝑔𝑟𝑖𝑑,𝑑,𝑒 . Evaporative water loss is water that exits through the cooling towers, which is proportional to the cooling load 𝐻𝑐𝑜𝑜𝑙,𝑑,𝑒 and the latent heat of vaporization 𝐽𝑤𝑎𝑡𝑒𝑟 : 𝐺 𝐸,𝑑,𝑒 = 𝐻𝑐𝑜𝑜𝑙,𝑑,𝑒 /𝐽𝑤𝑎𝑡𝑒𝑟 . CRACs require a coolant (typically water) for cooling, which must be treated when pollutant concentration reaches a threshold 𝜙 [24]. The treatment requires the polluted water to be sent to a blowdown-to-water treatment facility: 𝐺𝑏𝑙𝑜𝑤,𝑑,𝑒 = 𝐺 𝐸,𝑑,𝑒 /(1−𝜙). Lastly, energy generation consumes water at location-dependent intensity 𝐺𝐼𝑑 , which is proportional to the underlying grid makeup. The variation between sources can be dramatic, with wind using only 0.2 L/kWh, whereas hydropower uses 67 L/kWh, most of which is lost to evaporation [25]. This gives a water usage of 𝐺𝑔𝑟𝑖𝑑,𝑑,𝑒 = 𝐸𝑡𝑜𝑡,𝑑,𝑒 × 𝐺𝐼𝑑 . The total water usage in an epoch across all datacenters is:
Each geo-distributed datacenter 𝑑 comprises 𝑁𝑑 nodes at a location. All datacenters that are present in the geo-distributed network are defined by 𝐷, where each datacenter is present at a unique location. Nodes are organized in standard hot/cold aisle layouts with conventional air cooling [21]. Nodes vary in GPU types and configurations. Each node 𝑛 contains identical GPUs whose memory is pooled to serve incoming requests. We consider the NVIDIA A100 and H100 as the GPUs present in the nodes. The number of GPUs per node can be 2, 4, or 8, allowing each node to host models of varying complexity.
3.3
(𝐸𝑡𝑜𝑡,𝑑,𝑒 × 𝑇𝑂𝑈𝑑,𝑒 )
𝑑
𝑖
3.2
𝐷 ∑︁
(6)
3.5
Carbon Emissions Model
Carbon emissions arise from electricity generation. This has two components: grid-based emissions 𝑍𝑔𝑟𝑖𝑑,𝑑,𝑒 and water-treatmentbased emissions 𝑍𝐺,𝑑,𝑒 . Grid-based emissions depend on carbon intensity 𝐶𝐼𝑑,𝑒 (kg of carbon per kWh produced), which depends on the energy sources in the underlying grid. This gives an emissions of: 𝑍𝑔𝑟𝑖𝑑,𝑑,𝑒 = 𝐶𝐼𝑑,𝑒 × 𝐸𝑡𝑜𝑡,𝑑,𝑒 Wastewater processing consumes electricity from the local grid. However, potable water creation 𝐸𝐼𝑝𝑜𝑡,𝑒 and wastewater treatment 𝐸𝐼 𝑤𝑎𝑠𝑡𝑒,𝑒 have different energy intensities [12]. Potable water production arises from the need to replace evaporated water and process blowdown water. The potable carbon emissions are: 𝑍 𝑝𝑜𝑡,𝑑,𝑒 = (𝐺𝑏𝑙𝑜𝑤,𝑑,𝑒 + 𝐺 𝐸,𝑑,𝑒 ) × 𝐸𝐼𝑝𝑜𝑡,𝑒 The operation of the wastewater treatment plants is considered in terms of how they process grid-based water usage. The wastewater carbon emissions are: 𝑍 𝑤𝑎𝑠𝑡𝑒,𝑑,𝑒 = 𝐺𝑔𝑟𝑖𝑑,𝑑,𝑒 × 𝐸𝐼 𝑤𝑎𝑠𝑡𝑒,𝑒 Water-associated carbon emissions at a datacenter is scaled by the carbon intensity:
(4)
The total datacenter IT energy over an epoch is the sum of contributions from all its server nodes: 𝑁𝑑 ∑︁ 𝐸𝐼𝑇 ,𝑑,𝑒 = 𝐸𝐼𝑇 ,𝑛,𝑒 (5) 𝑛
Datacenter energy also includes cooling energy 𝐸𝑐𝑜𝑜𝑙,𝑑,𝑒 and infrastructure energy 𝐸𝑖𝑛𝑓 𝑟𝑎,𝑑,𝑒 . Cooling energy consists of the energy required to run the computer room air conditioning (CRAC) units. CRAC efficiency (COP) varies by datacenter, 𝐶𝑂𝑃𝑑 . Across an epoch 𝑒, the CRAC energy is 𝐸𝐶𝑅𝐴𝐶,𝑑,𝑒 = 𝐸𝐼𝑇 ,𝑑,𝑒 /𝐶𝑂𝑃𝑑 . Since chiller energy usage and auxiliary equipment energy usage are approximately equivalent to 𝐸𝐶𝑅𝐴𝐶,𝑑,𝑒 [22], the total datacenter cooling energy is: 𝐸𝑐𝑜𝑜𝑙,𝑑,𝑒 = 3 × 𝐸𝐶𝑅𝐴𝐶,𝑑,𝑒 . The supporting infrastructure consists of power distribution units and power supply units in the datacenter. Infrastructure
𝑍𝐺,𝑑,𝑒 = (𝑍 𝑝𝑜𝑡,𝑑,𝑒 + 𝑍 𝑤𝑎𝑠𝑡𝑒,𝑑,𝑒 ) × 𝐶𝐼𝑑,𝑒 3
(9)
Figure 2: Overview of MARLIN framework and its two phases. The total carbon emissions in a single epoch are the sum of all sources across all datacenters 𝐷: 𝑍𝑡𝑜𝑡,𝑒 =
𝐷 ∑︁
(𝑍𝑔𝑟𝑖𝑑,𝑑,𝑒 + 𝑍𝐺,𝑑,𝑒 )
(10) weighted voting to determine the final blended scheduling plan 𝑎˜ and agent-wise capital [𝐶 𝑗 ] 𝐽𝑗=1 . The final blended plan 𝑎˜ is deter′ mined by the agent-wise plan 𝑎 ∗𝑗 , capital threshold 𝛿 𝑗 , and utility function 𝑄 𝑗 . The capital threshold 𝛿 𝑗 determines when the agent 𝑗 can update the final plan according to its interest. 𝑄 𝑗 estimates the long-term importance of each proposal (vs. the short term reward ˜ each agent 𝑗 maintains a in phase 1). On top of the blended plan 𝑎, reserve of capital 𝐶 𝑗 that is updated after each epoch in the game. Capital 𝐶 𝑗 represents the influence of each agent when constituting the blended plan. We formulate phase 2 as a weighted resourceallocation game Γ which is defined as:
𝑑
4
Problem Formulation
We assume a single cloud service provider (e.g., Meta) that manages its geo-distributed datacenters for serving incoming LLM inference requests. Requests originate remotely and thus incur associated transfer times that must also be considered. Our problem optimization objective is to minimize the weighted sum of TTFT, carbon emissions, water usage, and energy costs over 𝐸 epochs, given resource constraints 𝑀𝑡𝑜𝑡,𝑛 and SLA constraints (SLA): 𝑀𝑖𝑛
𝐸 ∑︁
(𝑤 1 𝐿𝐴𝑡𝑜𝑡,𝑒 + 𝑤 2𝑍𝑡𝑜𝑡,𝑒 + 𝑤 3𝐺𝑡𝑜𝑡,𝑒 + 𝑤 4𝐶𝑜𝑠𝑡𝑡𝑜𝑡,𝑒 )
𝑒
˜ [𝐶 𝑗 ] 𝐽𝑗=1 = Γ([𝑎 𝑗 , 𝛿 𝑗 , 𝐶 𝑗 , 𝑄 𝑗 ] 𝐽𝑗=1 ) 𝑎,
𝑠.𝑡 .𝑀𝑛,𝑒 ≤ 𝑀𝑡𝑜𝑡,𝑛 ∀𝑛 ∈ 𝑁 𝐿𝐴𝑡𝑜𝑡,𝑒 ≤ 𝐿𝐴, ∀𝑖 ∈ 𝐼𝑒 4 ∑︁
MARLIN combines both phases with workload predictions and environmental state information to output a final blended plan 𝑎˜ for sustainable geo-distributed LLM inference scheduling. The following subsections describe the workload predictor, phases 1 and 2, and associated complexity analysis.
𝑤 𝑗 = 1, 𝑤 𝑗 ≥ 0
𝑗=1
Our goal is to develop a geo-distributed scheduling framework that assigns each incoming LLM inference request to a datacenter while optimizing the above objective and satisfying the constraints mentioned. Once a request is assigned to a datacenter, a local load balancer algorithm distributes requests to nodes within the datacenter. All datacenters use a modified round-robin load balancer based on [26], which ensures that nodes within a datacenter are not asymmetrically overwhelmed with requests.
5
(12)
(11)
5.1
Workload Predictor
LLM request volume varies over time and across locations, as shown in Fig. 1, necessitating accurate prediction to optimize resource allocations. By forecasting workloads, MARLIN can prewarm containers and avoid startup delays. We adopt and pretrain the predictor (line 1 in Algorithm 1) from [27], which utilizes a regression-based approach. The predictor forecasts the number of LLM requests in the next epoch based on information from past epochs within a time window of 𝑡𝑤, using an exponentially weighted moving average to estimate the incoming workload. This results in an extremely fast prediction time of roughly 100 microseconds per prediction. To evaluate the accuracy of the predictor it is compared to a baseline neural network predictor [27]. The regression-based predictor achieves over 90% accuracy in empirical validation experiments across various workload intensities and volatilities, demonstrating an average accuracy improvement of 19.01% over the baseline.
MARLIN Framework
Fig. 2 provides an overview of the MARLIN framework, which comprises agents that participate in a two-phase competitive game. To effectively service LLM requests, a scheduling plan 𝑎 is needed to pair requests with a datacenter to process them. In phase 1, each ′ agent 𝑗 independently proposes a single scheduling plan 𝑎 ∗𝑗 that best optimizes its single objective (e.g., TTFT, carbon, water, cost). ′ The quality of all plans 𝑎 ∗𝑗 in phase 1 is estimated through their respective reward value 𝑟 𝑗 . Each agent maintains its own reward function to score the plans. In phase 2, all 𝐽 agents negotiate via 4
5.2
Phase 1: Agent Training
In phase 1, we use a multi-agent soft actor-critic (SAC) algorithm with feature-wise linear modulation (FiLM) layer [28] and hindsight experience replay (HER) [29], as shown in Fig. 2. SAC is an actorcritic variant that uses stochastic off-policy learning to improve exploration. We add a FiLM layer to the actor’s MLP for improved feature modulation. HER is a technique that modifies the replay buffer labels and enables training in such a complex environment. The details of phase 1 are presented in Algorithm 1. The algorithm takes initial state information for the current epoch 𝑆𝑡𝑎𝑡𝑒𝑒 , pretrained predictor 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑜𝑟 , number of agents 𝐽 , agent-wise weights [𝑤 𝑗 ] 𝐽𝑗=1 , the past epochs’ request volume [𝐼𝑒 −1, ..., 𝐼𝑒 −𝑡 𝑤 ], and the initial policy parameter 𝜃 𝑗 as input for agent training. After SAC training with the help of FiLM layer and ′ HER, the algorithm outputs agent-wise plan 𝑎 ∗𝑗 . The 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑜𝑟 forecasts the current epoch’s LLM workload (lines 1-2). This becomes part of the state information 𝑆𝑡𝑎𝑡𝑒𝑒 of the current epoch, which also includes information about each datacenter’s node availability, carbon intensity, water intensity, and energy pricing. All agents are trained in parallel and iterate over 𝐾𝑜𝑝𝑡 iterations to update their policy parameter and Q function (lines 3-4). In each iteration, the agent 𝑗 will first take an action 𝑎 𝑗 as a scheduling plan and then apply feature-wise linear modulation on ′ 𝑎 𝑗 for a modulated plan 𝑎 𝑗 (lines 5-6). The FiLM layer applies affine transformations across the neural network input. This integrates workload data and datacenter attributes into a single input. Based ′ on the modulated plan 𝑎 𝑗 the optimization metrics are estimated (line 7) to be 𝑚𝑒𝑡𝑟𝑖𝑐 𝑗 = [𝐿𝐴𝑡𝑜𝑡 , 𝑍𝑡𝑜𝑡 , 𝐺𝑡𝑜𝑡 , 𝐶𝑜𝑠𝑡𝑡𝑜𝑡 ] 𝑎 𝑗 ′ We calculate the reward value 𝑟 𝑗 for modulated plan 𝑎 𝑗 (line 8) by considering an exponential moving average (𝐸𝑀𝐴), 𝑚𝑒𝑡𝑟𝑖𝑐 𝑗 , an ecological bonus that rewards consolidation 𝐸𝐶𝑂, and an SLA 𝑝𝑒𝑛𝑎𝑙𝑡𝑦 that punishes TTFT beyond a set point in the reward value estimation equation which is 𝑟 𝑗 = 𝐸𝑀𝐴 + 𝐸𝐶𝑂 +𝑚𝑒𝑡𝑟𝑖𝑐 𝑗 − 𝑝𝑒𝑛𝑎𝑙𝑡𝑦. The SAC training experience is stored in the replay buffer 𝐵𝑟𝑒𝑝𝑙𝑎𝑦,𝑗 (line 9). After 𝐾𝑜𝑝𝑡 iterations, agent 𝑗 generates the best plan 𝑎 ∗𝑗 by exploiting the policy parameter 𝜃 𝑗 (line 11). We apply FiLM to the ′ best plan 𝑎 ∗𝑗 to obtain the best modulated plan 𝑎 ∗𝑗 (line 12). Based ′ on the modulated plan 𝑎 ∗𝑗 , we estimate 𝑚𝑒𝑡𝑟𝑖𝑐 𝑗 through simulation (line 13). To balance current-epoch learning with historical insights, we maintain a cross-epoch buffer 𝐵𝑐𝑟𝑜𝑠𝑠,𝑗 for each agent 𝑗. The cross-epoch buffer is updated per epoch by sampling from the replay buffer 𝐵𝑟𝑒𝑝𝑙𝑎𝑦,𝑗 with HER (line 14).
5.3
based on each agent’s capital 𝐶 𝑗 (line 6). Moreover, an aggregate Q function 𝑄𝑡𝑜𝑡 is formulated to estimate the long-term quality of the blended plan 𝑎˜ (line 7). The blended plan is refined over 𝐾𝑜𝑝𝑡 iterations via stochastic gradient descent (SGD) [30] to maximize the aggregate Q function (lines 8-10). After each SGD step, a new blended plan is created via the critic weights (line 11). To maintain a balanced trade-off among all agents, a veto mechanism is designed based on the individual rationality (IR) theory [31]. IR is a game theory approach in which a participant will engage only if their expected payoff exceeds that of not participating. This holds for the veto mechanic, as agents are not required to veto in every single epoch and will only consider using their veto under the following circumstances: if 1) their capital exceeds a threshold 𝐶𝑡ℎ𝑟𝑒𝑠ℎ and 2) their critic network 𝑄 𝑗 estimates that the consensus provides a utility loss 𝜃 𝑗 more than 𝜃 𝑡ℎ𝑟𝑒𝑠ℎ compared to its own proposal (lines 12-14). If a veto scenario is triggered, we calculate the veto strength 𝑉 𝑒𝑡𝑜 𝑗 based on utility value degradation 𝜃 𝑗 and maximal veto strength 𝑉 𝑒𝑡𝑜 𝑗 (line 15). Based on the veto strength, we update the blended plan 𝑎˜ by moving the blended plan toward ′ the agent’s plan 𝑎 ∗𝑗 (line 16). Once all agents are satisfied with the blended plan (no threshold 𝐶𝑡ℎ𝑟𝑒𝑠ℎ or 𝜃 𝑡ℎ𝑟𝑒𝑠ℎ is violated), the optimization metric values 𝑚𝑒𝑡𝑟𝑖𝑐 are estimated accordingly via simulation (line 19). To pass down the knowledge we learn from the veto process, we need to update the capital 𝐶 𝑗 used in the veto mechanism for the next epoch. We first calculate the performance score 𝑃𝑒𝑟 𝑓 𝑗 for each agent’s capital 𝐶 𝑗 (line 21). The performance score estimates how well the current blended plan 𝑎˜ serves the agent’s 𝑚𝑒𝑡𝑟𝑖𝑐 𝑗 . After that, the bonus score 𝐵𝑜𝑛𝑢𝑠 𝑗 is calculated to estimate each agent’s contribution
Phase 2: Competitive Proposals ′
Phase 2 focuses on combining the separate scheduling plans 𝑎 ∗𝑗 ˜ We employ generated from phase 1 into a single blended plan 𝑎. game theory to optimize this blending process. The agent’s associated capital 𝐶 𝑗 is essential in advancing its own plan in the final blended solution. Algorithm 2 presents the details of phase 2. First, we calculate the utility value 𝑞 𝑗 for each agent 𝑗 (lines 1-2). After that, we calculate the aggregated utility value 𝑞𝑡𝑜𝑡 for all agents 𝐽 (line 4 in Algorithm 2). The weight of each agent’s plan in the blended plan is determined by the corresponding utility value 𝑞 𝑗 scaled by the aggregated utility value 𝑞𝑡𝑜𝑡 (line 5). To help formulate the blended plan, we calculate the initial critic weight 𝜔 𝑗 5
Figure 3: Comparison of the (a) TTFT, (b) carbon emissions, (c) water usage, and (d) energy costs, across frameworks.
6
Figure 4: Comparison of the PHV values across LLM inference scheduling frameworks. to the final blended plan (line 22). Given the performance score 𝑃𝑒𝑟 𝑓 𝑗 , bonus score 𝐵𝑜𝑛𝑢𝑠 𝑗 , predefined growth rate 𝜂, and predefined scaling factor 𝛽, we update all the capitals using a bounded EMA approach (line 23).
5.4
Experimental Results
To empirically validate our framework, MARLIN, we compare it against eight state-of-the-art workload scheduling algorithms. These can be divided into five heuristic-based approaches (Helix [13], Splitwise [14], NSGA-II [32], PerLLM [15], and SLIT [16]), and three RL methods (QLearning [33], DDQN [34], and ActorCritic [35]). Datacenters are distributed globally with requests originating uniformly across all regions. Each datacenter has 1000 compute nodes across 6 uniformly distributed node types, each containing 2, 4, or 8 NVIDIA A100 or H100 GPUs for LLM inference on Xeonbased server nodes. We developed and validated a Python simulator implementing the models from Section 3, as well as the workload predictor and MARLIN algorithmic framework. Data for computational energy and LLM runtime is obtained from profiling on real server nodes and integrated into the simulator. We consider five schemes to be run through our framework, which consist of four plans dominated by one agent (MARLINMinCarbon, -MinWater, -MinCost, -MinLatency) and one with equal weights (MARLIN-Balanced). Our baseline experimental setup spans 24 hours with the maximum datacenter utilization rate in an epoch at 95% across 8 datacenters. The upcoming epoch’s workload is forecast (a 15-minute window) in the previous epoch. The learning rate hyperparameter in the MARLIN framework was 0.0003 for the actor and 0.001 for the critic networks. We used a gamma of 0.95 and a tau of 0.005 for the agents. The actor network has 128 hidden dimensions, while the critic was increased to 256 to capture the complexity of the design space. The replay buffer size was 20,000 samples for the current epoch and 5,000 samples for previous epochs, with a 30/70 sampling split. For the SGD step, we used a learning rate of 0.05 with 5 steps. To determine the strength and use of the veto, the required threshold was 150 capital, with a 0.5 pull towards the agent’s own plan when used.
Complexity Analysis
The runtime and memory complexity of the framework depend on the number of datacenters 𝐷, policy network parameters 𝜃 𝑗 , number of agents 𝐽 , replay buffer size 𝐵𝑟𝑒𝑝𝑙𝑎𝑦,𝑗 , and exploration steps 𝐾𝑜𝑝𝑡 . In phase 1, the runtime complexity is driven by exploration steps 𝐾𝑜𝑝𝑡 and the policy network parameters 𝜃 𝑗 . Agents 𝐽 are run in parallel. Therefore, the final runtime complexity is 𝑂 (𝐾𝑜𝑝𝑡 × 𝜃 𝑗 ). The memory complexity of phase 1 relies on the size of the replay buffers 𝐵𝑟𝑒𝑝𝑙𝑎𝑦,𝑗 as well as the policy network parameters 𝜃 𝑗 . The buffer and the policy network sizes both grow with the total agents 𝐽 . The memory complexity of phase 1 is therefore 𝑂 (𝐽 × 𝜃 𝑗 + 𝐽 × 𝐵𝑟𝑒𝑝𝑙𝑎𝑦,𝑗 × 𝐷). As the iterations of the blended plan are performed across one plan, the runtime complexity of phase 2 is identical to phase 1 as 𝑂 (𝐾𝑜𝑝𝑡 ×𝜃 𝑗 ). The memory complexity of phase 2 is much lower than that of phase 1. The only two memory requirements are the holding ′ of the proposals 𝑎 ∗𝑗 from phase 1 and the agents 𝐽 themselves. The proposal footprint from phase 1 is proportional to the number of datacenters 𝐷. Therefore, the memory complexity of phase 2 is 𝑂 (𝐽 × 𝜃 𝑗 + 𝐽 × 𝐷).
6.1
State-of-the-art Comparison
Fig. 3 shows MARLIN’s performance compared with state-of-theart frameworks across the TTFT, Carbon, Water, and Energy Cost metrics, with confidence bounds for all RL frameworks. The MARLIN variants outperform all comparison works across these metrics, even considering confidence bounds. For the single-metric schemes, MARLIN-MinLatency reduces TTFT by 18.67% over QLearning, MARLIN-MinCarbon reduces carbon emissions 33.61% more than QLearning, MARLIN-MinWater reduces water 43.88% more than 6
Figure 5: Comparison of (a) TTFT, (b) carbon emissions, (c) water usage, and (d) energy costs across LLM inference scheduling frameworks as the number of datacenters changes between 4 and 12; shaded regions represent the confidence intervals. DDQN, and MARLIN-MinCost reduces costs 11.72% more than ActorCritic. MARLIN-Balanced outperforms the heuristic approaches across all metrics except TTFT and outperforms the RL baselines on 2 of 4 metrics; it is at most 23.61% slower in TTFT and 2.77% higher in carbon emissions than QLearning, while achieving at least an 18.66% reduction in water (vs. DDQN) and 18.94% cost reduction (vs. ActorCritic). Compared to the only other explicitly sustainable framework, SLIT, MARLIN-Balanced achieves a 2.29% reduction in TTFT, 61.07% in carbon, 81.11% in water, and 60.90% in total costs — demonstrating that MARLIN succeeds as a sustainable framework where SLIT failed. In Fig. 4, we summarize the Pareto hypervolume (PHV) values across the different frameworks. The PHV of a Pareto solution front (the front formed by the non-dominated solutions) is a metric that measures the hypervolume of the solution space dominated by the set of solutions. This is an effective measure of the quality of diverse solution sets generated by a multi-objective optimization framework. A higher PHV indicates a more diverse and higher quality solution set. For comparison works that produce only a single point, the PHV is calculated as the geometric volume of the four-dimensional hyperrectangle defined by that point. We set a reference point for the worst observed metrics. The population of points along the Pareto front was 1 for heuristic frameworks. For the RL frameworks, we archived the best points generated during the search along the confidence interval providing between 10 and 15 points. MARLIN’s Pareto front contained 40 points, well exceeding the value of the other frameworks. We observe a significant difference between the heuristic-based and the RL-based approaches. The best-performing heuristic framework is SLIT, which covers only 40.09% of the hypervolume covered by MARLIN. The RL frameworks perform much better, but still only cover at most 68.27% of the volume that MARLIN covers. These results show that MARLIN is able to search the complex solution space much more effectively than the other frameworks, which allows it to provide higher quality solutions.
search space, as datacenter counts increased. The RL-based approaches have more success in utilizing the new datacenters and their different carbon and water intensities. MARLIN variants dominate, with MARLIN-MinCarbon being an outlier as it outperformed MARLIN-MinCost on cost with 12 datacenters. This performance by the carbon agent highlights a limitation of the single-metric variants. Low carbon sources can be low cost, but the signal in the carbon case is stronger than the cost case. MARLIN-MinCarbon finds this strong signal early and takes advantage of low carbon and cheap electricity for longer than MARLIN-MinCost, leading to the discrepancy. Crucially, the MARLIN-Balanced approach can achieve Paretooptimal solutions as the search space grows. This is significant, MARLIN-Balanced can reduce water usage by an average of 16.4% compared to smaller search spaces. MARLIN-Balanced leverages the unique sustainability fingerprint of each datacenter it has access to. In this experiment, we observe the same trend between MARLINBalanced and SLIT. In 12 datacenters, MARLIN-Balanced reduces latency by 4.66%, carbon emissions by 65.22%, water usage by 81.7%, and cost by 56.59%. MARLIN-Balanced provides a more balanced solution instead of the best across all metrics.
We analyzed the contribution of each component of the MARLIN framework by selectively removing them in an ablation study. The experiments use baseline parameters. The results of the ablation study are shown in Fig. 6. The plot shows that the PHV improvement of the MARLIN (full_baseline) against the best-performing ablation is at least 14.6%. We can observe that all parts of the MARLIN framework are important towards improving the quality of the generated solutions. For instance, without the least impacting component Veto, the PHV of MARLIN will decrease 14.9%. Without the blending process in phase 2, the PHV will decrease 66.5%.
6.2
7
6.3
Scalability Analysis
To observe how the frameworks behave as the problem size scales, we vary the number of datacenters from 4 to 12. The results of this experiment are presented in Fig. 5. It can be observed that the heuristic approaches were unable to utilize the new datacenters effectively and to navigate the larger
MARLIN Framework Ablation Study
Conclusion
In this work, we presented a novel framework for serving LLM inference requests across geo-distributed cloud datacenters, called MARLIN. We empirically demonstrated that MARLIN can optimize the TTFT of LLM inference requests while also optimizing sustainability metrics such as carbon emissions and water usage. Solutions 7
Architectural Support for Programming Languages and Operating Systems, Volume 1, ASPLOS ’25, (New York, NY, USA), p. 586–602, Association for Computing Machinery, 2025. [14] P. Patel et al., “Splitwise: Efficient generative llm inference using phase splitting,” in 2024 ACM/IEEE 51st ISCA, pp. 118–132, 2024. [15] Z. Yang et al., “Perllm: Personalized inference scheduling with edge-cloud collaboration for diverse llm services,” 2024. [16] H. Moore, S. Qi, N. Hogade, D. Milojicic, C. Bash, and S. Pasricha, “Sustainable carbon-aware and water-efficient llm scheduling in geo-distributed cloud datacenters,” in Proceedings of the GLSVLSI 2025, GLSVLSI ’25, (New York, NY, USA), p. 929–934, Association for Computing Machinery, 2025. [17] Y. Wang et al., “Burstgpt: A real-world workload dataset to optimize llm serving systems,” in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD ’25, (New York, NY, USA), p. 5831–5841, Association for Computing Machinery, 2025. [18] H. Touvron et al., “Llama 2: Open foundation and fine-tuned chat models,” 2023. [19] T. Y. Cheng and X. Jia, “Delay-sensitive multicast in inter-datacenter wan using compressive latency monitoring,” IEEE Transactions on Cloud Computing, vol. 8, no. 1, pp. 86–96, 2020. [20] P. G. Recasens et al., “Mind the memory gap: Unveiling gpu bottlenecks in large-batch llm inference,” in 2025 IEEE 18th CLOUD, pp. 277–287, 2025. [21] R. F. Sullivan, “Alternating cold and hot aisles provides more reliable cooling for server farms,” White Paper, Uptime Institute, 2000. [22] Q. Zhang et al., “A survey on data center cooling systems: Technology, power consumption modeling and control strategy optimization,” Journal of Systems Architecture, vol. 119, p. 102253, 2021. [23] K. M. U. Ahmed et al., “A review of data centers energy consumption and reliability modeling,” IEEE Access, vol. 9, pp. 152536–152563, 2021. [24] M. A. B. Siddik et al., “The environmental footprint of data centers in the united states,” Environmental Research Letters, vol. 16, p. 064017, may 2021. [25] Y. Jin et al., “Water use of electricity technologies: A global meta-analysis,” Renewable and Sustainable Energy Reviews, vol. 115, p. 109391, 2019. [26] M. Kushwaha et al., “Advanced weighted round robin procedure for load balancing in cloud computing environment,” in 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pp. 215–219, 2021. [27] V. Mittal et al., “Mu: An efficient, fair and responsive serverless framework for resource-constrained edge clouds,” in Proceedings of the ACM Symposium on Cloud Computing, SoCC ’21, (New York, NY, USA), p. 168–181, Association for Computing Machinery, 2021. [28] E. Perez et al., “Film: Visual reasoning with a general conditioning layer,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, 2018. [29] M. Andrychowicz et al., “Hindsight experience replay,” in Advances in Neural Information Processing Systems (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.), vol. 30, Curran Associates, Inc., 2017. [30] S. Ruder, “An overview of gradient descent optimization algorithms,” 2017. [31] J. Von Neumann and O. Morgenstern, “Theory of games and economic behavior, 2nd rev,” 1947. [32] K. Deb et al., “A fast and elitist multiobjective genetic algorithm: Nsga-ii,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182–197, 2002. [33] C. Guo et al., “Q-learning-based workload consolidation for data centers with composable architecture,” IEEE Transactions on Industrial Informatics, vol. 21, no. 3, pp. 2324–2333, 2025. [34] L. Alsalem and K. Djemame, “Task scheduling in edge computing environments: a hierarchical cluster-based federated deep reinforcement learning approach,” in Proceedings of the 18th IEEE/ACM International Conference on Utility and Cloud Computing, UCC ’25, (New York, NY, USA), Association for Computing Machinery, 2026. [35] T. Hidayat et al., “Reinforcement learning-driven hybrid precopy/postcopy vm migration for energy-efficient data centers,” IEEE Access, vol. 13, pp. 169521– 169533, 2025.
Figure 6: Normalized PHV of MARLIN framework (full_baseline) compared to various ablations. generated by the MARLIN framework outperformed other stateof-the-art frameworks and produced a balanced scheduling plan when configured with equal metric weights. This was observed in our baseline experiments, where MARLIN, decreased TTFT by 18%, carbon emissions by 33%, water usage by 43%, and costs by 11%. MARLIN is designed to operate as a meta scheduler above containers that host LLMs, such as Kubernetes or vLLM. This allows MARLIN to be used by datacenter managers to provide a range of LLM inference services and enable sustainable, multi-objective decision-making. Our future work will explore integrating embodied carbon to optimize life-cycle carbon emissions for LLM serving.
Acknowledgments This research was made possible with support from HPE and grants from the National Science Foundation (CCF-2324514, CNS-2132385).
References [1] A. Muhammad, “Llm statistics 2026: Comprehensive insights into market trends and integration,” 2026. Accessed on Mar.31.2026. [2] S. Singh, “Chatgpt users statistics (2026) – active users & global growth data,” 2026. Accessed on Mar.31.2026. [3] Y. Li et al., “The unseen ai disruptions for power grids: Llm-induced transients,” 2024. [4] R. Leppert, “What we know about energy use at u.s. data centers amid the ai boom,” 2026. Accessed on Mar.31.2026. [5] N. Jegham et al., “How hungry is ai? benchmarking energy, water, and carbon footprint of llm inference,” 2025. [6] T. Xiao et al., “Environmental impact and net-zero pathways for sustainable artificial intelligence servers in the usa,” Nature Sustainability, pp. 1–13, 2025. [7] C. He, Z. Liu, J. Wu, et al., “Future global urban water scarcity and potential solutions,” Nature communications, vol. 12, no. 1, p. 4667, 2021. [8] N. Hogade, S. Pasricha, and H. J. Siegel, “Energy and network aware workload management for geographically distributed data centers,” IEEE Transactions on Sustainable Computing, vol. 7, no. 2, pp. 400–413, 2022. [9] R. Chen et al., “Power and thermal-aware virtual machine scheduling optimization in cloud data center,” Future Generation Computer Systems, vol. 145, pp. 578– 589, 2023. [10] S. Qi, H. Moore, N. Hogade, D. Milojicic, C. Bash, and S. Pasricha, “Casa: A framework for slo- and carbon-aware autoscaling and scheduling in serverless cloud computing,” in 2024 IEEE 15th IGSC, pp. 1–6, 2024. [11] B. M. Beena et al., “A green cloud-based framework for energy-efficient task scheduling using carbon intensity data for heterogeneous cloud servers,” IEEE Access, vol. 13, pp. 73916–73938, 2025. [12] S. Qi, H. Moore, D. Milojicic, C. Bash, and S. Pasricha, “Shield-eb: Sustainable hybrid evolutionary-boosting framework for carbon, wastewater, and cost-aware datacenter management,” IEEE Access, vol. 14, pp. 40878–40898, 2026. [13] Y. Mei et al., “Helix: Serving large language models over heterogeneous gpus and network via max-flow,” in Proceedings of the 30th ACM International Conference on 8