arXiv CS open access

Execution-Feasible Dispatch of LLM Pre-training Loads for Renewable-Dominant AI Data Centers

Xi Tong et al. · arxiv_cs

arXiv CS · Papers

cloud distributed-computing parallel-computing

AI data centers (AIDCs) are emerging as large power-system loads, making their operational flexibility increasingly valuable. Existing studies often treat AIDC demand as continuously adjustable or generically delay-tolerant, which overstating the flexibility of large language model (LLM) pretraining

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

Execution-Feasible Dispatch of LLM Pre-training Loads for Renewable-Dominant AI Data Centers

arXiv:2610.13047v1 [cs.DC] 9 Oct 2026

Xi Tong, Ershun Du*, Member, IEEE, Haiyang Jiang, Member, IEEE, Long Chen, Member, IEEE, Qin Wang*, Senior Member, IEEE

Abstract—AI data centers (AIDCs) are emerging as large power-system loads, making their operational flexibility increasingly valuable. Existing studies often treat AIDC demand as continuously adjustable or generically delay-tolerant, which overstating the flexibility of large language model (LLM) pretraining. This paper proposes an execution-feasible dispatch framework for LLM pre-training in renewable-dominant AIDCs. LLM descriptors are first converted into checkpoint-domain daily workloads, then mapped to feasible 3D-parallel GPU modes with explicit throughput, GPU-capacity, memory, and IT-power limits. The resulting checkpoint- and mode-constrained load states are embedded into a day-ahead power dispatch model. Case studies on a 2048 GPUs AIDC with two LLaMA pretraining tasks show that checkpoint-aware 3D dispatch reduces operating cost by 20.02% and renewable curtailment from 1.077 to 0.014 MWh relative to fixed continuous training. Related results also show that checkpoint records alone do not create load flexibility,practical grid-interactive flexibility arises from checkpoint waiting combined with execution-feasible 3D-mode compensation. Index Terms—AI data center, LLM pre-training load, execution-feasible dispatch, checkpoint-aware flexibility, 3D parallelism, renewable-dominant AIDC.

I. I NTRODUCTION A. Background and Motivation RTIFICIAL intelligence (AI) data centers are becoming new class of large electric loads in modern power systems. The rapid deployment of large language models (LLMs), foundation models, and GPU-accelerated services is increasing both the scale and the temporal concentration of the demand for data-center electricity [1]–[4]. Meanwhile, high renewable penetration requires more flexibility on the demand-side to mitigate renewable uncertainty, curtailment, and peak-stress conditions. Recent studies indicate that AI data centers (AIDCs) can potentially act as grid-interactive assets through software-defined workload orchestration, demand response, and power scheduling [5], [6]. Therefore, AIDCs should not be modeled only as passive high-power consumers, their computing-side flexibility should be converted into power system readable load states.

Corresponding authors: Ershun Du and Qin Wang. X. Tong and Q. Wang are with the Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China.(e-mail:polyu-eee-xi.tong@connect.polyu.hk, qinee.wang@polyu.edu.hk.) E. Du is with the Lab of Low Carbon Energy, Tsinghua University, Beijing, China.(e-mail:duershun@tsinghua.edu.cn) H. Jiang is with the Department of Electrical Engineering, Tsinghua University, Beijing, China.(e-mail:jiang hy@tsinghua.edu.cn) L. Chen is with Tsinghua University and the Department of Computing, The Hong Kong Polytechnic University, China.(e-mail:longchen.cs@ieee.org)

However, this conversion is nontrivial for LLM pre-training. Most existing data center load models describe computing demand as aggregate power representations or workload-shifting abstractions [7]–[10]. Recent studies have further refined data center load flexibility through optimal energy consumption characteristics, service level agreements (SLAs), or geographically dispersed operation [11]–[14]. These abstractions are useful for conventional Internet services and cloud batch jobs, but they may overestimate the flexibility of LLM pretraining. LLM pre-training is a sequential optimization process composed of a large number of training iterations. Its temporal shift is constrained by checkpoint states, while its resumption power level is determined by discrete feasible GPU parallel configurations rather than by a continuous power command [15], [16]. In addition, memory feasibility, GPU capacity, parallel efficiency, and deadline-constrained training progress jointly determine whether delayed workload can be recovered. Thus, the key challenge is not simply to shift AIDC demand, but to derive an execution-feasible AIDC load model that can participate in power dispatch. B. Literature Review The energy footprint and grid impact of data centers have been widely studied. Global data center energy analyzes show that efficiency improvements have historically mitigated a part of demand growth, while the rapid expansion of cloud services and AI accelerators still makes data centers important electricity consumers [1]–[3]. For AI workloads, the energy and carbon implications of training large neural models and foundation models have been reported in recent studies [4], [17], [18]. These works motivate low-carbon and grid-aware AIDC operation, but they mainly quantify energy or carbon impacts and do not provide execution-feasible load models for power system dispatch. Data center demand response and green operation have also been investigated from the perspectives of workload shifting, local generation, microgrid operation, and renewable-powered operation [8], [19]–[21]. Other studies consider electricity cost optimization in green data centers [12]–[14], [22]. Integrated energy-system studies further couple data centers with smart grids, storage, and on-site renewable generation [23]–[26]. More recent studies refine data center flexibility by considering carbon-aware computing, QoS-assured demand response, and dispatchable capacity evaluation [27]–[30]. Multi-data-center coordination has also been studied for tie-line power smoothing and cross-site operation [31]. These studies establish data centers as useful demand-side resources for cost reduction,

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

peak management, and renewable absorption. Nevertheless, most formulations still rely on aggregate flexible-demand envelopes, divisible workload models, or utilization-based power models. Such representations cannot explicitly determine when an LLM training load can be safely paused, what discrete power state it will take after resumption, and whether the delayed progress can be completed before the deadline. The computer systems literature provides the missing execution mechanisms of LLM training. Large-scale Transformer training is enabled by tensor parallelism, pipeline parallelism and data parallelism [15], [16], [32], [33]. Memory pressure is addressed through optimizer-state partitioning, sublinear-memory checkpointing, and tensor re-materialization [34]–[36]. Activation recomputation is further refined for large Transformer training [37]. Recent carbon-aware machine learning (ML) cluster schedulers also show that job shifting and resource scaling can reduce carbon footprint or peak power at the cluster level [38]. However, these techniques are mainly designed for throughput, memory efficiency, or cluster workload orchestration. Current research on the interaction between LLM pre-training and power system is still scarce. C. Contributions and Paper Organization Although current literature offers robust strategies for workload shifting and multi-site coordination, these general approaches do not directly translate into an execution-feasible load model for LLM pre-training. In particular, these approaches often overlook three parts: the safe temporal boundaries for load interruption, the discrete power profiles upon task resumption, and the feasibility of deadline-constrained progress recovery. To fill this gap, this paper develops an execution-feasible AIDC dispatch framework that converts checkpoint states and feasible three-dimensional (3D) parallel GPU modes into the day-ahead power dispatch model. The main contributions are summarized as follows. • An execution-feasible LLM pre-training load model is developed by integrating checkpoint progress and feasible 3D-parallel GPU modes. • A checkpoint-state graph based dispatch method is proposed to preserve GPU, memory, throughput, and deadline feasibility during load shifting. • The system value of execution-feasible LLM load flexibility is quantified in a renewable-dominant AIDC, revealing its cost-saving, curtailment-reduction, and BESScomplementary effects. The remainder of this paper is organized as follows. Section II introduces the AIDC system framework and the flexibility mechanism of LLM pre-training loads. Section III presents the execution-feasible load modeling method. Section IV formulates the day-ahead power dispatch model. Section V reports the numerical case study, and Section VI concludes the paper. II. S YSTEM F RAMEWORK AND P ROBLEM D ESCRIPTION A. Framework of Green-Power Direct Supply AIDC System Fig. 1 shows the considered green-power direct-supply AIDC. Local wind and photovoltaic generation are preferentially delivered to the AIDC through a dedicated or equivalent

Power Side: Green Electricity and Energy Storage Green Electricity Base

Computing Side: AIDC Training Execution Layer

Large Language Model Pre-Training Tasks

LLM-1

LLM-2

…

LLM-n

Energy Storage

Power Grid

Power Supply

Limited GPU Resource Pool

Other IT power

Fig. 1. Architecture of the green-power direct-supply AIDC system.

green-power interface, while the upstream grid provides reliability support and time-of-use price signals. Unlike islanded operation, renewable generation, grid purchase, curtailment, and BESS operation are jointly coordinated to satisfy AIDC demand [20], [21]. On the computing side, the AIDC electric load consists of non-schedulable base IT power and schedulable LLM pretraining power. The external electric load is obtained from the internal IT power through the power usage effectiveness (PUE) [39]: PtAIDC = PUE Ptbase,IT + Pttrain,IT , ∀t ∈ T , (1) where Ptbase,IT is the basic IT power, and Pttrain,IT is the power of active LLM pre-training tasks. The green-power bus balance is Ptren,av −Ptcur +Ptgrid +Ptdis = PtAIDC +Ptch ,

∀t ∈ T , (2)

where Ptren,av , Ptcur , Ptgrid , Ptch , and Ptdis are available renewable power, renewable curtailment, grid purchase, BESS charging, and BESS discharging, respectively. B. Flexibility Description of LLM Pre-Training Loads Fig. 2 contrasts conventional flexible loads with LLM pretraining loads. Conventional data-center models often use continuously adjustable power, delay-tolerant jobs, or utilizationbased envelopes [9], [10], while LLM pre-training can safely wait or resume only at checkpoint states. Waiting reduces instantaneous training power but does not reduce remaining computation. After resumption, the power level is discrete and determined by feasible GPU execution modes involving tensor, pipeline, and data parallelism, memory feasibility, throughput, and GPU capacity [15], [16]. Thus, LLM pretraining flexibility is a set of checkpoint-constrained and mode-discrete execution states, where checkpoints provide temporal boundaries and feasible 3D modes provide recovery capability [35], [37]. The task–computing–resource mapping chain is train,IT Ωtask −→ Cj , Sjckpt −→ Aeff −→ PtAIDC , j j −→ Pt (3) where Ωtask is the LLM task descriptor, C is the daily training j j workload, Sjckpt is the checkpoint-domain progress set, and Aeff j is the pruned feasible 3D-parallel mode set.

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

where Nj , Dj , Bj , and sj are model size, full token target, global batch size, and sequence length; lj , hj , hf,j , nh,j , and Vj are Transformer architecture parameters; rj is the daily tranche ratio; and Tjdead is the deadline.

Conventional Flexible Load

Power Load

Concrete continuity, non-consideration, practical arithmetic behavior

tend

tstart

B. Task-to-Computing Mapping and Checkpoint-Domain Progress

For a dense causal-decoder Transformer, the per-iteration training FLOPs can be approximated according to commonly used Transformer FLOPs accounting in LLM training [16]. The per-iteration workload is written as Vj sj 2 + , (5) Fiter,j = χF B s l h 1 + j j j j j 6hj 16lj hj

Execution-constrained LLM Pre-training C0

Checkpoint

Can stop pre-training

Medium GPUs High GPUs

None GPU

Low GPUs

Power Load

Power Load Reshaping Pre-training with high/medium/low GPUs

Wait during high price/low renewable energy——training stop AIDC basic load

tstart

tend

Fig. 2. Conceptual flexibility of LLM pre-training loads for power dispatch.

C. Coordination Framework and Dispatch Problem Based on this mechanism, Fig. 3 gives the proposed coordination framework. The task layer describes model size, token target, batch size, sequence length, and deadline. The computing layer converts these descriptors into daily workload and checkpoint progress, and the resource layer maps them into feasible 3D GPU modes with different throughput, GPU occupation, memory usage, and IT power. The dispatch layer then coordinates these execution-feasible load states with renewable generation, grid purchase, curtailment, and BESS operation under checkpoint-state continuity, completion, GPU-capacity, AIDC-capacity, BESS, and power-balance constraints. III. M ODELING OF LLM P RE - TRAINING L OADS A. LLM Training Workflow and Task Descriptor Fig. 4 shows the macro-level LLM workflow and the iteration-level distributed training loop. The proposed model focuses on iterative pre-training and periodic checkpointing; data preparation, validation, model export, and post-training stages are not scheduled. Forward/backward computation and distributed communication determine workload and feasible execution modes, while checkpoints provide safe pause/resume states for power dispatch [37]. For each LLM pre-training job j ∈ J , the task descriptor is defined as Ωtask = Nj , Dj , Bj , sj , lj , hj , hf,j , nh,j , Vj , rj , Tjdead , j (4)

where χF j is the training FLOPs coefficient; the three terms in parentheses capture dense matrix operations, attention-related computation, and vocabulary projection. The coefficient can absorb FLOPs-counting conventions and implementation overheads. The number of full-task iterations and the number of dailytranche iterations are Dj , (6a) Ij = Bj sj Ijday = ⌈rj Ij ⌉ .

(6b)

Accordingly, the daily-tranche computing workload is Cj = Ijday Fiter,j .

(7)

Let Sj,t ∈ [0, 1] be the normalized progress of the daily tranche at time interval t. The remaining computing workload is rem Cj,t = (1 − Sj,t )Cj .

(8)

To represent safe waiting states, the daily tranche is divided into Kjseg aggregated checkpoint-to-checkpoint segments. The checkpoint-domain index set and progress set are ( ) k seg ckpt Kj = 0, 1, . . . , Kj , Sj = k ∈ Kj . (9) Kjseg The workload of one aggregated checkpoint segment is Cjseg =

Cj . Kjseg

(10)

Checkpoint waiting produces no training progress, but the required computation is conserved: X Xj,t ∆t = Cj , (11) t∈T

where Xj,t is the realized training throughput in FLOPs/s. Checkpointing does not reduce total computation; it only enables temporal reshaping if delayed workload can be recovered by a faster feasible mode.

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

Task Layer

Computing Layer

Resource Layer

LLM Task Descriptor

Daily Training Workload

Feasible 3D Parallel Pattern Generation

Iterative FLOPs, Daily Training Computation, Remaining Workload, et al.

Model Scale N Target Token D

Pipeline Parallelism

Tensor Parallelism

Checkpoint Domain Progress

Batch Size B

Transformer Layer L

...

Memory Constraint Divisibility Constraint

Effective Pattern Set Aeff

GPU limit

k Data Parallelism

Only allowed to wait at checkpoints

...

Power Dispatch Layer

Time Flexibility： Checkpoint waiting

IT Power

Day-ahead Optimization

Outputs

Checkpoint State Design

AIDC Load Curve

Wait Edge & Training Edge

GPU Scheduling

Pattern Selection

Electricity Purchase

Grid Electricity Purchase

Storage Trajectory

Wind and Solar Curtailment

Training Completion

Power Balance： Source-Grid-Load-Storage Collaboration

Resource Compensation： 3D Parallel Recovery

Fig. 3. Task–computing–resource coordination framework.

(a)

(b)

𝑣1

𝑣2 𝑣3

𝑣4

𝑢1

cution mode, which determines GPU occupation, throughput, and training IT power. A candidate 3D-parallel configuration is defined as

𝑢2

a = (pa , qa , da ),

(12a)

na = pa qa da ,

(12b)

…

𝑢3.1

𝑢3.𝑚

𝑣5

𝑢4

𝑣6

𝑢5

𝑣7

…

𝑢6.1

𝑣8

𝑣9

𝑢6.𝑚

𝑢8

𝑢7

Fig. 4. Workflow of (a) LLM training and (b) Iterative training. Three basic parallel strategies

3D Parallelism (Model Partitioning)

A Pipeline Parallelism Layer 1

Pipeline Parallelism

Stage 1

…

GPU Group 1

Stage 2

…

GPU Group 2

Layer 2

…

Stage p

Layer L

…

Layer L-1

…

GPU Group p

Tensor Parallelism t

B Tensor Parallelism

…

W(1)

W(2)

GPU 1

GPU 2

W(t)

C Data Parallelism

(𝟏)

𝒙𝟏

(𝟏)

𝒙𝟐

… 𝒙(𝟏) 𝑩

GPU t 𝒆𝒇𝒇

𝒙𝟏

(𝟐)

𝒙𝟏

(𝟐)

𝒙𝟐

Effective Mode Set 𝑨𝒋

… 𝒙𝑩

𝒙𝟐

… 𝒙(𝟐) 𝑩

Data Group 1

Data Group 2

…

(𝒅)

𝒙𝟏

…

(𝒅)

𝒙𝟐

GPU Number

Training Throughout

IT Power

GPU Memory

… 𝒙(𝒅) 𝑩

Data Group d …

Xj,a = na Rg ηj,a ,

Fig. 5. Basic parallel strategies, 3D-parallel mode definition, and effective resource attributes for LLM training.

C. Computing-to-Resource Mapping and 3D-Parallel Execution Mode Fig. 5 illustrates the computing-to-resource mapping. Pipeline, tensor, and data parallelism jointly form a 3D exe-

X̄j,a = Xj,a ∆t.

(13)

where Xj,a is in FLOPs/s; X̄j,a is in FLOPs per interval; and ηj,a is the efficiency factor, which can be obtained from cluster profiling records or calibrated performance models; in the numerical study, it is used as a representative mode attribute to capture pipeline bubbles, communication, synchronization, and implementation losses. Memory feasibility is screened by state Mj,a =

d Data Parallelism 3D Mode Definition 𝒂 = (𝒑𝒂 , 𝒒𝒂 , 𝒅𝒂 ), 𝒏𝒂 = 𝒑𝒂 𝒒𝒂 𝒅𝒂

…

where pa , qa , and da are the pipeline-, tensor-, and dataparallel sizes; na is the required number of GPUs. The 3D mode links model partitioning with GPU occupation and training power in the AIDC dispatch model. Given the per-GPU peak FLOP rate Rg and the end-toend efficiency ηj,a , the effective training throughput and the compute amount within one scheduling interval are

ωs Nj , pa qa ζastate

act Mj,a = ρrec j

(14a)

lj Bj hj Bj hf,j λh sj + λf sj pa da q a da qa ! Bj nh,j + λa s2j da qa

Bj hj Bj V j sj + λv sj , (14b) da qa da qa GPU state act state act Mj,a = Mj,a + Mj,a + M0 + δ other Mj,a + Mj,a . (14c) + λo

where ωs is the mixed-precision model-state memory coefficient; ζastate is the state-partitioning factor, with ζastate = 1

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

is the activationif no state partitioning is used [34]; ρrec j recomputation coefficient; λh , λf , λa , λo , and λv are calibrated activation coefficients; and M0 and δ other are fixed and proportional runtime memory overheads. A mode is memory-feasible only if GPU Mj,a ≤ Mgcap .

(15)

In addition to memory feasibility, 3D-parallel execution must satisfy engineering divisibility conditions: mod(lj , pa ) = 0,

(16a)

mod(hj , qa ) = 0,

mod(hf,j , qa ) = 0,

(16b)

mod(nh,j , qa ) = 0,

mod(Bj , da ) = 0.

(16c)

These constraints ensure valid layer assignment, tensor/head partitioning, and batch splitting. The IT power of mode a is modeled as IT comm Pj,a = na Pgtrain + Pj,a ,

Algorithm 1 Feasible 3D-Parallel Configuration Generation and Pruning Require: Task descriptor Ωtask ; GPU parameters Rg and Mgcap ; candidate j max . sets P, Q, and D; GPU limit NGPU Ensure: Effective mode set Aeff . j 1: Initialize Aj ← ∅. 2: for all p ∈ P, q ∈ Q, and d ∈ D do 3: Set a = (p, q, d) and na = pqd. 4: if the divisibility conditions in (16) are violated then 5: Continue to the next candidate. 6: end if 7: Compute Xj,a and X̄j,a according to (13). GPU according to (14). 8: Compute Mj,a IT according to (17). 9: Compute Pj,a GPU ≤ M cap and n ≤ N max then 10: if Mj,a a g GPU 11: Add a and ϕj,a to Aj . 12: end if 13: end for 14: Remove dominated modes from Aj according to (20). 15: Return the remaining modes as Aeff j .

(17)

Benchmark training: continuous training Training throughput

where Pgtrain is the equivalent active training power per GPU; comm Pj,a is the communication and mode-dependent overhead. comm In the case study, Pj,a is absorbed into the equivalent mode

Pre-training completion status

Checkpoint 1 Checkpoint 2 Checkpoint 3 Checkpoint k

Training progress 100%

𝑿𝟎

90%

…

power coefficient.

Training Start 𝒕𝟎

𝒕𝟏

𝒕𝟐

Training End 𝒕𝐞𝐧𝐝

𝒕𝒌

𝒕𝟑

Time

20%

Training Section 1

Training Section 2

D. Feasible Mode Set and Dominance Pruning

Training Section k

𝒕𝟎

𝒕𝟏

𝒕𝒌 𝒕𝐞𝐧𝐝

𝒕𝟐

Time

Checkpoint-aware training: interrupt scheduling

For each job j, the feasible 3D-parallel mode set is defined as GPU max Aj = a Mj,a ≤ Mgcap , na ≤ NGPU , (18) (16) is satisfied . Each feasible mode is defined by IT GPU ϕj,a = na , Xj,a , X̄j,a , Pj,a , Mj,a .

Training Section 3

Training throughput

Pre-training completion status

Checkpoint 1 Checkpoint 2 Checkpoint 3 Checkpoint k

Training progress

X𝐡𝐢𝐠𝐡

100%

𝑿𝟎

90% 85%

…

𝑿𝐥𝐨𝐰

Training Start 𝒕𝟎

𝒕𝟏

𝒕𝟐

𝒕𝟑

Training End 𝒕𝐞𝐧𝐝

𝒕𝒌

Time

20% 15% 10% 0

Training Section 1

(19)

The tuple ϕj,a gives the GPU occupation, throughput, interval workload, IT power, and memory requirement of one executable mode. To reduce the number of online dispatch choices, a dominance pruning rule is applied. Mode a is dominated by mode b if IT IT nb ≤ na , Pj,b ≤ Pj,a , Xj,b ≥ Xj,a , (20) with at least one strict inequality. A dominated mode consumes no less GPU resource and IT power while providing no higher throughput, and can be removed. The remaining effective mode set is Aeff j . These modes are execution-feasible candidates under the adopted hardware and model assumptions, rather than proprietary production configurations of a specific LLM training system. Algorithm 1 summarizes their generation and pruning. IV. C HECKPOINT-AWARE DAY-A HEAD G REEN -P OWER D ISPATCHING M ODEL A. Mode-Pruned Checkpoint-State Graph Fig. 6 compares fixed continuous training with checkpointaware dispatch. In the benchmark case, the job is executed with a fixed throughput X0 . In the proposed case, state changes occur only at checkpoint boundaries; low/high-computing segments correspond to feasible 3D modes in Aeff j , and pauses

Training Section 2

Baseline training

Training Section 3

Training Section k

Low-computing

𝒕𝟎

𝒕𝟏

𝒕𝒌 𝒕𝐞𝐧𝐝

𝒕𝟐

Training pause

Time

High computing

Fig. 6. Checkpoint-aware interruptible dispatch mechanism compared with benchmark continuous training.

correspond to checkpoint waiting. Incompatible mode transitions can be excluded during edge generation. Let T = {1, 2, . . . , T } be the set of dispatch intervals and H = {0, 1, . . . , T } be the set of time-state nodes. For each training job j, a checkpoint-state graph Gj = (Vj , Ej ) is constructed as Vj = {(τ, k) | τ ∈ H, k ∈ Kj } ∪ {gj },

(21)

where (τ, k) is the state in which job j has reached checkpoint k at the beginning of time state τ , and gj is an artificial sink node. The source node is sj = (0, 0). Two types of edges are introduced. The first type is the waiting edge: e : (τ, k) → (τ + 1, k),

τ = 0, . . . , T − 1,

(22)

which means that job j waits for one dispatch interval at checkpoint k. The second type is the training edge: e : (τ, k) → (τ + ℓj,a,∆k , k + ∆k), Aeff j

(23)

where a ∈ is the selected 3D-parallel mode, ∆k is the number of checkpoint segments continuously executed

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

by this edge, 1 ≤ ∆k ≤ K jump , and k + ∆k ≤ Kjseg . The parameter K jump is the maximum number of checkpoint segments represented by one training edge. For a training edge e, the executed workload and integer duration are ∆k edge = seg Cj , Cj,e (24a) Kj & edge ' Cj,e . (24b) ℓj,a,∆k = X̄j,a The active slot set of edge e is Γe = {τ + 1, τ + 2, . . . , τ + ℓj,a,∆k }.

(25) Tjdead .

A training edge is generated only if τ + ℓj,a,∆k ≤ Finally, zero-power completion edges connect every terminal checkpoint node (τ, Kjseg ) with τ ≤ Tjdead to the sink node gj . Let xj,e ∈ {0, 1} indicate whether edge e is selected. The selected edges must form one feasible path from sj to gj . The path constraints are written in a unified net-flow form: X X xj,e − xj,e = 1, (26a) e∈δj− (sj )

e∈δj+ (sj )

xj,e −

xj,e = 0,

∀v ∈ Vj \ {sj , gj },

e∈δj− (v)

e∈δj+ (v)

(26b) X e∈δj− (gj )

xj,e −

xj,e = 1.

(26c)

Eq. (28a) gives the schedulable training IT power formed by active GPU modes, while (28b) gives the corresponding GPU occupation. Together with the non-schedulable base IT power, the total AIDC electric load is PtAIDC = PUE Ptbase,IT + Pttrain,IT , ∀t ∈ T . (29) Thus, selected edges determine active 3D modes, active modes determine training IT power, and PUE converts IT power into total AIDC load. The GPU-capacity and AIDC power-capacity constraints are max Gtrain ≤ NGPU , t

∀t ∈ T ,

PtAIDC ≤ P AIDC,max ,

(30a)

∀t ∈ T .

(30b)

For reporting purposes, the normalized progress of job j by the end of interval t can be decoded from the completed training edges: Sj,t =

X 1 ∆k(e)xj,e , seg Kj tr

∀t ∈ T ,

(31)

e∈Ej : τ̄ (e)≤t

where τ̄ (e) is the ending time state of edge e. This decoded progress variable is not required for feasibility because the path constraints already ensure that each job reaches the terminal checkpoint before its deadline.

e∈δj+ (gj )

where δj+ (v) and δj− (v) are the outgoing and incoming edge sets of node v, respectively. Constraints (26) enforce checkpoint-state continuity and daily-tranche completion. Since waiting edges only start from checkpoint nodes, non-checkpoint interruption is excluded by graph construction. The ceiling operation in (24b) conservatively counts the last partially utilized slot as occupied, avoiding underestimation of the time and power required to complete a checkpoint segment.

For each selected training edge, let a(e) be the associated 3D-parallel mode and let ∆k(e) be the number of checkpoint segments completed by the edge. The mode-use indicator is linearly decoded as X yj,a,t = xj,e , ∀j, a, t, (27)

where Ejtr is the set of training edges. Because the selected path is time-monotonic, at most one training edge can be active for a given job in a given time interval. The aggregate training IT power and GPU usage are X X IT Pttrain,IT = Pj,a yj,a,t , (28a) j∈J a∈Aeff j

j∈J a∈Aeff j

Ptren,av − Ptcur + Ptgrid + Ptdis = PtAIDC + Ptch ,

∀t ∈ T . (32) Renewable curtailment and grid purchase are bounded by 0 ≤ Ptcur ≤ Ptren,av ,

∀t ∈ T ,

na yj,a,t .

(28b)

(33a)

∀t ∈ T .

(33b)

The BESS state is indexed on H, while the charging and discharging powers are indexed on T . The BESS dynamics, power limits, SOC limits, and charge/discharge exclusiveness are bat Etbat = Et−1 + η ch Ptch ∆t −

e∈Ejtr : a(e)=a, t∈Γe

Gtrain = t

The power-balance interface introduced in Section II is imposed as an operational constraint at the green-power bus:

0 ≤ Ptgrid ≤ P grid,max ,

B. Edge-to-Power Decoding and Resource Constraints

X X

C. Green-Power Bus and BESS Operation Constraints

Ptdis ∆t, η dis

∀t ∈ T , (34a)

ch,max 0 ≤ Ptch ≤ uch , ∀t ∈ T , t P dis dis dis,max 0 ≤ Pt ≤ ut P , ∀t ∈ T , ch dis ch dis ut + ut ≤ 1, ut , ut ∈ {0, 1}, min E ≤ Etbat ≤ E max , ∀t ∈ H, ETbat = E0bat .

(34b) (34c) ∀t ∈ T ,

(34d) (34e) (34f)

For no-BESS scenarios, Ptch = Ptdis = 0, and the BESS state equations are removed.

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

1600

Power/kW

seg Require: {Ωtask }, {Aeff j j }, {Kj }, renewable and price forecasts, BESS and AIDC parameters. Ensure: Training paths, 3D-mode schedules, AIDC load, grid purchase, curtailment, and BESS operation. 1: for all training job j ∈ J do 2: Initialize the checkpoint-state graph Gj = (Vj , Ej ) by (21). 3: Add checkpoint waiting edges by (22). jump , and (23)– 4: Generate feasible training edges using Aeff j , ∆k ≤ K (25). IT 5: Attach edge attributes {a(e), ∆k(e), Γe , na(e) , Pj,a(e) } and remove infeasible edges violating deadlines. 6: Add zero-power completion edges to the sink node gj . 7: end for 8: Construct the path-selection MILP with (26). 9: Decode selected edges into training power, GPU usage, and AIDC load by (27)–(29). 10: Add resource, green-power bus, and BESS constraints (30)–(34). 11: Solve (36) and recover training paths, progress, load, grid purchase, curtailment, and BESS trajectories.

(a) RE Wind PV

1200 800 400 0

Time/h

(b) 0.08

Price/USD∙kWh-1

Algorithm 2 Checkpoint-Aware Training Graph Construction and Dispatch

0.07 0.06 0.05 0.04 0.03

Time/h

Fig. 7. Case-study input: (a) forecasting output of wind and solar generation; (b) time-of-use electricity price.

D. Objective Function and Solution Procedure TABLE I R EPRESENTATIVE LLM P RE - TRAINING PARAMETERS

The day-ahead dispatch objective is X grid grid X min J = πt Pt ∆t + ccur Ptcur ∆t t∈T

t∈T

cbat Ptch + Pt

dis

Parameter

(35) ∆t.

t∈T

The three terms are upstream-grid purchase cost, renewablecurtailment opportunity cost, and BESS throughput cost. The GPU pool is assumed to be a dedicated pre-training pool, so the opportunity cost of reallocating GPUs to external services is not included in the base objective. The complete optimization model is min

s.t.

(26), (36)

(32) − (34), dis uch t , ut ∈ {0, 1},

LLM-A

LLM-B

LLaMA-13B 13.0B/1.0T 40/5120/13824/40 2048/32000 135168 (4, 4, 32)/512 4.900% 12/24 h

LLaMA-33B 32.5B/1.4T 60/6656/17920/52 2048/32000 530432 (6, 4, 32)/768 2.021% 12/24 h

TABLE II K EY AIDC AND P OWER -S YSTEM PARAMETERS Parameter

(27) − (30), xj,e ∈ {0, 1},

Representative model Model size/training tokens Layers/h/hf /heads Sequence length/vocabulary Reported GPU-hours Baseline (p, t, d)/GPUs Daily ratio Segments/deadline

GPU type Physical (GB) Base IT load (kW) PV/wind (kW) BESS efficiency

Value A100 80 375 1050/350 0.95

Parameter

Value

Peak FLOPS/GPU 312 TFLOPS PUE 1.25 Power/GPU (kW) 0.40 BESS kW/kWh 300/800 Curt./BESS cost 20.83USD/MWh

Ptgrid , Ptcur , Ptch , Ptdis , Etbat ≥ 0. The model is a MILP. Memory, divisibility, and 3D-mode engineering feasibility are embedded in Aeff j by Algorithm 1, AIDC while yj,a,t , Pttrain,IT , Gtrain , and P are linearly decoded t t from selected checkpoint-state paths. Algorithm 2 summarizes the procedure. V. N UMERICAL C ASE S TUDY A. Basic Data and Case Settings Considering a green-power direct-supply AIDC with 2048 GPUs, we assume that this AIDC is only used for LLM pretraining tasks. Renewable-generation profiles and electricity prices are selected from CAISO. The profiles are matched to the case capacities in Table II. Fig. 7 shows the renewable and TOU-price inputs, where fixed continuous training is not naturally aligned with renewable availability or electricity price.

We choose two representative LLaMA pre-training tasks, as showen in Table I . Model architecture, token targets, nominal batch, and GPU-hours follow public data [40]. The implemented batch, baseline modes, checkpoint segments, and daily tranches are case settings; FFN dimensions and daily compute are derived. Table II lists the case designed capacities and costs. The AIDC load follows from the base load, GPU power, pool size, and PUE [39]. 1) Validation Scenarios: Six scenarios are designed to isolate the effects of checkpoint information, 3D-mode variability, and BESS operation, as shown in Table III. S1 is the fixed baseline. S2 only records checkpoint states along the S1 trajectory and does not allow waiting, so S1 and S2 are intentionally identical. S3 is an idealized 3D-only benchmark without checkpoint continuity. S4 is the executable checkpoint-aware 3D case, S5 isolates BESS under fixed training, and S6 is the full coordination case.

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

Checkpoint

3D modes

S1: Fixed-NoCkpt-NoBat S2: CkptRecord-Fixed3D-NoBat S3: 3D-NoCkpt-NoBat S4: Ckpt-3D-NoBat S5: Fixed-Bat S6: Ckpt-3D-Bat

No No No No Yes Yes

No Yes No Yes No Yes

Fixed Fixed Variable Variable Fixed Variable

B. Overall Scenario Comparison Table IV compares the six scenarios. S1 is the fixed continuous-training baseline, whereas S2 only records checkpoint states along the same fixed trajectory. Their identical results show that checkpoint information alone cannot reshape the load unless checkpoint waiting and subsequent recovery actions are enabled. S3 achieves the lowest cost among the nostorage cases, reducing cost by 21.99% and eliminating curtailment. However, it allows idealized 3D-mode variation without enforcing checkpoint-state continuity and should therefore be interpreted as an optimistic benchmark rather than a physically executable policy. In contrast, S4 enforces checkpoint-state continuity and still reduces cost by 20.02% while lowering renewable curtailment from 1.077 to 0.014 MWh relative to S1. Its cost is only 9.34 USD above that of S3, so it retains approximately 91% of the idealized cost saving under explicit execution constraints. The storage comparison further clarifies the source of flexibility. S5 reduces cost by only 5.45% and requires 1.442 MWh of BESS throughput because the training trajectory remains fixed. S6 combines checkpoint-aware 3D dispatch with BESS, reducing cost by 20.13%, eliminating curtailment, and requiring only 0.026 MWh of BESS throughput. The additional cost reduction from S4 to S6 is only 0.14%, indicating that executable workload reshaping provides the dominant benefit, while BESS corrects the small residual source–load mismatch. C. Source–Load–Storage–Grid Dispatch Comparison Fig. 8 compares the 24-h power balance of S1 and S6. In S1, the two LLM profiles follow fixed continuous-training trajectories, and the AIDC load therefore remains at 1015 kW throughout the day. This rigid demand cannot increase to absorb renewable surpluses or decrease when renewable output falls and the electricity price rises. Consequently, S1 purchases 8.965 MWh from the grid and curtails 1.077 MWh of renewable energy, with a renewable utilization rate of 93.46% and a green-power share of 63.20%. In S6, checkpoint waiting and feasible 3D-mode selection reshape the AIDC load between 455.0 and 1335.2 kW. The scheduler raises training demand during renewable-rich periods and lowers it in the late evening, while BESS performs only a brief residual charge–discharge adjustment. This coordination eliminates renewable curtailment, reduces grid purchase from 8.965 to 8.422 MWh, and increases the green-power share from 63.20% to 66.17%. Thus, the proposed method does not simply reduce electricity consumption; it redistributes

1600 1200 800 400 0

Time/h

(b) 1600

Power/kW

BESS

1200 800 400 0 0

Time/h Curtailment

AIDC load

Renewable used

Grid purchase

BESS

Fig. 8. Source–load–storage–grid balance of representative scenarios: (a) S1 fixed continuous training; (b) S6 checkpoint-aware 3D dispatch with BESS. (a)

(b)

Feasible modes

Effective modes

High GPU memory

Low GPU memory

Fig. 9. Feasible 3D-parallel configuration space and retained effective modes: (a) LLM-A-13B; (b) LLM-B-33B. 2000

AIDC load LLM-B LLM-A Base load

1600

Power/kW

Scenario

Power/kW

(a)

TABLE III S CENARIO D EFINITIONS

1200 800 400 0

Time/h

Fig. 10. AIDC load composition under the full S6 dispatch model.

executable training work in time to align demand with renewable availability, with storage serving as a complementary balancing resource. D. Mechanism Analysis of Checkpoint-Aware 3D Training Dispatch Fig. 9 shows the discrete operating modes available to the scheduler. After memory, divisibility, and GPU-capacity screening, 135 feasible configurations are generated for LLMA and 111 for LLM-B. Pareto pruning retains 12 and 21 effective modes, respectively, whereas the optimal S6 schedule

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

TABLE IV OVERALL S CENARIO P ERFORMANCE Scenario S1 S2 S3 S4 S5 S6

Main mechanism

Cost (USD)

Cost red. (%)

Grid (MWh)

Curt. (MWh)

BESS thru. (MWh)

RE util. (%)

Green share (%)

Peak power (kW)

Peak GPU

Fixed baseline Checkpoint record Idealized 3D only Checkpoint+3D Fixed+BESS Checkpoint+3D+BESS

473.03 473.03 368.99 378.33 447.23 377.79

0.00 0.00 21.99 20.02 5.45 20.13

8.965 8.965 8.395 8.434 8.281 8.422

1.077 1.077 0.000 0.014 0.319 0.000

0.000 0.000 0.000 0.000 1.442 0.026

93.46 93.46 100.00 99.92 98.06 100.00

63.20 63.20 66.24 66.12 66.00 66.17

1015.0 1015.0 1399.0 1335.2 1015.0 1335.2

1280 1280 2048 2048 1280 2048

(a) GPU numbers

2400

Total GPUs LLM-B LLM-A

1800 1200 600 0 0

(b) 120%

Training progress

Time/h LLM-A LLM-B

feasibility at 447.23 USD. Most further savings are obtained by 1536 GPUs, after which cost approaches 377.79 USD at 2048. GPUs are therefore most valuable when they first enable post-waiting recovery. In Fig. 12(c), increasing the renewableto-AIDC ratio from 0.75 to 1.375 lowers cost from 557.84 to 197.94 USD and raises green share from 49.66% to 86.69%. At 1.5, green share reaches 89.14%, but cost rises slightly and curtailment reaches 2.127 MWh. This reversal marks the point at which additional renewable energy exceeds the absorption capability of workload and storage flexibility.

80%

VI. C ONCLUSION

40% 0%

Time/h

Fig. 11. GPU allocation and daily training progress under the full S6 dispatch model: (a) GPU usage; (b) training progress.

activates only four modes for LLM-A and three for LLMB. These retained modes provide distinct GPU–throughput– power combinations that allow delayed training work to be recovered without introducing arbitrary continuous load scaling or carrying the full configuration space into the dispatch problem. Figs. 10 and 11 verify that the resulting electrical trajectory is execution-feasible. GPU allocation reaches the 2048-GPU cluster limit during recovery, and the corresponding AIDC power varies from 455.0 to 1335.2 kW. LLM-A does not pause and continues training in a 160-GPU low-power mode during the late-evening interval, whereas LLM-B waits at a checkpoint for three hours and resumes at hour 22. Its cumulative progress remains unchanged during waiting and subsequently recovers through feasible 3D modes. Both profiles complete 100% of their daily compute by hour 24 with zero boundary idle. The load steps in Fig. 10 therefore correspond to explicit checkpoint and 3D-mode decisions, confirming that the reported power-system benefits are obtained without violating training progress or execution constraints.

This paper proposed an execution-feasible dispatch framework for representative LLM pre-training load profiles in a grid-connected green-power direct-supply AIDC. The proposed framework maps LLM descriptors to checkpoint-domain workloads and feasible 3D-parallel GPU modes, and embeds the retained execution modes into a checkpoint-state graph coupled with renewable curtailment, grid purchase, and BESS operation. Numerical studies on a 2048-GPU AIDC with two representative LLaMA-derived profiles show that checkpointaware 3D dispatch reduces the operating cost by 20.02% and decreases renewable curtailment from 1.077 MWh to 0.014 MWh compared with fixed continuous training. When BESS is further coordinated, the full dispatch case reduces the operating cost by 20.13%, eliminates renewable curtailment, and requires only 0.026 MWh of BESS throughput. The results confirm that checkpoint information alone does not create load flexibility; useful flexibility appears only when checkpoint waiting is combined with feasible 3D-mode compensation. In the studied case, workload reshaping provides the dominant flexibility, while BESS performs residual balancing. Note that the numerical profiles and generated 3D modes are constructed from public trace and data, rather than proprietary production traces. Future work will extend the framework to multi-day stochastic dispatch and other AI workloads such as fine-tuning and inference, include idle-GPU opportunity cost under shared GPU pools, and investigate coordinated dispatch among multiple AIDCs.

E. Sensitivity Analysis Fig. 12(a) shows diminishing checkpoint value. The final cost improvement is only 1.51%; 12 checkpoints segments capture most of the available flexibility. Fig. 12(b) identifies a GPU feasibility threshold. Capacities of 768–1152 GPUs cannot complete both tranches, while 1280 GPUs restore

R EFERENCES [1] E. Masanet, A. Shehabi, N. Lei, et al., “Recalibrating global data center energy-use estimates,” Science, vol. 367, no. 6481, pp. 984–986, 2020. [2] J. Koomey and E. Masanet, “Does not compute: Avoiding pitfalls assessing the Internet’s energy and carbon impacts,” Joule, vol. 5, no. 7, pp. 1625–1628, 2021.

IEEE TRANSACTIONS ON SMART GRID, VOL. XX, NO. X, XXX 2026

Fig. 12. Sensitivity analysis of key dispatch parameters: (a) checkpoint granularity; (b) GPU capacity; (c) renewable integration.

[3] A. de Vries, “The growing energy footprint of artificial intelligence,” Joule, vol. 7, no. 10, pp. 2191–2194, 2023. [4] D. Patterson, J. Gonzalez, U. Hölzle, et al., “The carbon footprint of machine learning training will plateau, then shrink,” Computer, vol. 55, no. 7, pp. 18–28, 2022. [5] P. Colangelo, A. K. Coskun, J. Megrue, et al., “AI data centres as gridinteractive assets,” Nature Energy, vol. 11, pp. 254–261, 2025. [6] L. Lin, R. Wijayawardana, V. Rao, et al., “Exploding AI power use: An opportunity to rethink grid planning and management,” in Proc. ACM Int. Conf. Future Energy Syst. (e-Energy), 2024, pp. 434–441. [7] A. Qureshi, R. Weber, H. Balakrishnan, et al., “Cutting the electric bill for Internet-scale systems,” in Proc. ACM SIGCOMM, 2009, pp. 123– 134. [8] Z. Liu, A. Wierman, Y. Chen, et al., “Data center demand response: Avoiding the coincident peak via workload shifting and local generation,” Performance Evaluation, vol. 70, no. 10, pp. 770–791, 2013. [9] M. Ghamkhari and H. Mohsenian-Rad, “Energy and performance management of green data centers: A profit maximization approach,” IEEE Trans. Smart Grid, vol. 4, no. 2, pp. 1017–1025, 2013. [10] J. Ma, R. Yao, B. Zhang, et al., “Data-driven flexibility capability modeling of Internet data center considering task dependency,” IEEE Internet Things J., vol. 11, no. 14, pp. 24538–24550, 2024. [11] Y. Zhou, F. Wei, S. Li, et al., “Data center load modeling through optimal energy consumption characteristics: A path to simultaneously enhance energy efficiency and demand response quality,” Applied Energy, vol. 393, 126095, 2025. [12] R. Basmadjian, J. F. Botero, G. Giuliani, et al., “Making data centres fit for demand response: Introducing GreenSDA and GreenSLA contracts,” IEEE Trans. Smart Grid, vol. 9, no. 4, pp. 3453–3464, 2018. [13] A. Kiani and N. Ansari, “Profit maximization for geographically dispersed green data centers,” IEEE Trans. Smart Grid, vol. 9, no. 2, pp. 703–711, 2018. [14] A. Pahlevan, M. Zapater, A. K. Coskun, et al., “ECOGreen: Electricity cost optimization for green datacenters in emerging power markets,” IEEE Trans. Sustain. Comput., vol. 6, no. 2, pp. 289–305, 2021. [15] M. Shoeybi, M. Patwary, R. Puri, et al., “Megatron-LM: Training multi-billion parameter language models using model parallelism,” arXiv:1909.08053, 2019. [16] D. Narayanan, M. Shoeybi, J. Casper et al., “Efficient large-scale language model training on GPU clusters using Megatron-LM,” in Proc. SC, 2021. [17] E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for deep learning in NLP,” in Proc. ACL, 2019, pp. 3645–3650. [18] A. S. Luccioni, S. Viguier, and A.-L. Ligozat, “Estimating the carbon footprint of BLOOM, a 176B parameter language model,” arXiv:2211.02001, 2022. [19] Z. Liu, M. Lin, A. Wierman, et al., “Greening geographical load balancing,” IEEE/ACM Trans. Netw., vol. 23, no. 2, pp. 657–671, 2015. [20] J. Li and W. Qi, “Toward optimal operation of Internet data center microgrid,” IEEE Trans. Smart Grid, vol. 9, no. 2, pp. 971–979, 2018. [21] I. Goiri, W. Katsak, K. Le, et al., “Parasol and GreenSwitch: Managing

datacenters powered by renewable energy,” in Proc. ASPLOS, 2013, pp. 51–64. [22] T. Yang, Y. Zhao, H. Pen, et al., “Data center holistic demand response algorithm to smooth microgrid tie-line power fluctuation,” Applied Energy, vol. 231, pp. 277–287, 2018. [23] C. Guo, F. Luo, Z. Cai, et al., “Integrated energy systems of data centers and smart grids: State-of-the-art and future opportunities,” Applied Energy, vol. 301, 117474, 2021. [24] X. Long, Y. Li, Y. Li, et al., “Collaborative response of data center coupled with hydrogen storage system for renewable energy absorption,” IEEE Trans. Sustainable Energy, vol. 15, no. 2, pp. 986–1000, 2023. [25] Y. Guo, Y. Gong, Y. Fang, et al., “Energy and network aware workload management for sustainable data centers with thermal storage,” IEEE Trans. Parallel Distrib. Syst., vol. 25, no. 8, pp. 2030–2042, 2014. [26] X. Cao, J. Zhang, and H. V. Poor, “Data center demand response with on-site renewable generation: A bargaining approach,” IEEE/ACM Trans. Netw., vol. 26, no. 6, pp. 2707–2720, 2018. [27] A. Radovanovic, R. Koningstein, I. Schneider, et al., “Carbon-aware computing for datacenters,” IEEE Trans. Power Syst., vol. 38, no. 2, pp. 1270–1280, 2023. [28] Y. Zhang, D. C. Wilson, I. C. Paschalidis, et al., “HPC data center participation in demand response: An adaptive policy with QoS assurance,” IEEE Trans. Sustain. Comput., vol. 7, no. 1, pp. 157–171, 2022. [29] A. Vafamehr, M. E. Khodayar, and S. D. Manshadi, “A framework for expansion planning of data centers in electricity and data networks under uncertainty,” IEEE Trans. Smart Grid, vol. 10, no. 1, pp. 305–316, 2019. [30] J. Han, E. Du, B. Du, et al., “Evaluating the dispatchable capacity of cross-regional data center clusters toward power system operation,” IEEE Trans. Smart Grid, vol. 17, no. 2, pp. 1180–1193, 2026. [31] T. Yang, X. Ma, S. Cai, et al., “Multi-data center tie-line power smoothing method based on demand response,” IEEE Trans. Cloud Comput., vol. 12, no. 4, pp. 983–995, 2024. [32] Y. Huang, Y. Cheng, A. Bapna, et al., “GPipe: Efficient training of giant neural networks using pipeline parallelism,” in Proc. NeurIPS, 2019. [33] A. Harlap, D. Narayanan, A. Phanishayee, et al., “PipeDream: Fast and efficient pipeline parallel DNN training,” in Proc. SOSP, 2019, pp. 1–15. [34] S. Rajbhandari, J. Rasley, O. Ruwase, et al., “ZeRO: Memory optimizations toward training trillion parameter models,” in Proc. SC, 2020. [35] T. Chen, B. Xu, C. Zhang, et al., “Training deep nets with sublinear memory cost,” arXiv:1604.06174, 2016. [36] P. Jain, A. Jain, T. N. D. Pham, et al., “Checkmate: Breaking the memory wall with optimal tensor rematerialization,” in Proc. MLSys, 2020. [37] V. Korthikanti, J. Casper, S. Lym, et al., “Reducing activation recomputation in large transformer models,” in Proc. MLSys, 2023. [38] K. Xu, D. Sun, H. Tian, et al., “GREEN: Carbon-efficient resource scheduling for machine learning clusters,” in Proc. USENIX NSDI, 2025, pp. 999–1014. [39] L. A. Barroso, U. Hölzle, and P. Ranganathan, The Datacenter as a Computer: Designing Warehouse-Scale Machines, 3rd ed. Morgan & Claypool, 2019. [40] H. Touvron, T. Lavril, G. Izacard, et al., “LLaMA: Open and efficient foundation language models,” arXiv:2302.13971, 2023.

Execution-Feasible Dispatch of LLM Pre-training Loads for Renewable-Dominant AI Data Centers

Analyze & explore related research in Conceptio

Related documents