JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
1
Enhancing Energy Efficiency in Scientific Workflows through CFD based PIVAEs
arXiv:2605.23850v1 [cs.DC] 22 May 2026
Ali Zahir, Ashiq Anjum, Mark Wilkinson and Jeyan Thiyagalingam Abstract—The growing complexity and scale of scientific workflows in high performance computing (HPC) environments have led to significant challenges in managing energy consumption without compromising computational performance. Traditional scheduling strategies often fail to account for the complex interplay between thermal dynamics, workload diversity, and system scalability—leading to inefficient and unsustainable energy usage. This paper introduces a novel, scalable, and AI-assisted scheduling framework for optimizing energy consumption in high-performance computing (HPC) environments without compromising performance. Central to our approach is the integration of Computational Fluid Dynamics (CFD) with a Physics-Informed Variational Autoencoder (PIVAE), enabling the generation of physically realistic synthetic workload data that bridges the gap between thermodynamic behavior and scheduler decision-making in complex, multi-scale HPC environments. By categorizing workflows based on resource utilization profiles, we evaluate the impact of multiple scheduling strategies such as Locality Aware and Speculative Aware Scheduling on system performance and energy efficiency.These workflows—ranging from event reconstruction to anomaly detection—represent diverse computational intensities, demonstrating the adaptability of the proposed method across varying scientific domains. Our results show that modest reductions in CPU performance (e.g., to 15%) can yield substantial energy savings (up to 10%) with only minor turnaround time increases (approximately 5–6%), identifying an optimal operational sweet spot. This work demonstrates how physics-informed generative modeling can enable adaptive, sustainable, and data-efficient scheduling decisions for next-generation HPC infrastructures. The proposed methodology offers a scalable path forward for sustainable data processing in modern distributed computing infrastructures. Index Terms—Cloud computing, Energy efficiency, Task scheduling, Intelligent scheduling, Data locality, Supervised learning, Scientific workflow, Dynamic threshold, Energy aware scheduling, Resource allocation, optimisation, and Performance metrics.
✦
1
I NTRODUCTION
L
ARGE-SCALE distributed computing systems, including high-performance computing (HPC) clusters and cloud platforms, form the backbone of modern scientific discovery in domains such as genomics, climate modeling, and high-energy physics, where complex, multi-stage computational workflows orchestrate data processing, simulation, and analysis tasks. These systems must process petabytes of data under stringent time constraints, while operating within limited power and cooling budgets. Energy inefficiency has therefore emerged as a first-order design constraint: studies estimate that CPU underutilization alone accounts for nearly 32% of the total energy draw in HPC environments [1], with additional waste stemming from thermal overheads, repetitive I/O, and imbalanced workload execution. Figure 1 illustrates the distribution of power consumption across typical computing system components, emphasizing the dominant role of CPU and cooling subsystems in overall energy draw. Despite continuous advancements in energy-efficient scheduling, existing approaches remain largely empirical and fail to integrate thermodynamic feedback into decision-making. Our work Manuscript received –; revised –. Ali Zahir is with the School of Computing and Mathematical Sciences, University of Leicester. (e-mail: [email protected] ). Ashiq Anjum (Prof.) is with the School of Computing and Mathematical Sciences, University of Leicester. (e-mail: [email protected] ). Mark I. Wilkinson (Prof.) is with the Department of Physics and Astronomy, University of Leicester. (e-mail: [email protected] ). Jeyan Thiyagalingam (Dr.) is with the Scientific Machine Learning Group, Rutherford Appleton Laboratory, Science and Technology Facilities Council (STFC–UKRI). (e-mail: [email protected] ).
introduces a physics-informed, AI-driven approach that unifies data generation and scheduling under a single, physically realistic framework—addressing both the scale and sustainability challenges of next-generation HPC systems. As operational costs and carbon footprints escalate, energyaware scheduling has become critical for both sustainability and performance. Task scheduling plays a pivotal role in determining the trade-offs between turnaround time (TAT) and energy efficiency. Traditional schedulers such as First Come First Serve (FCFS) and Round Robin emphasize fairness and simplicity but do not account for heterogeneous hardware, thermal dynamics, or workload diversity [2]. More advanced heuristics such as Particle Swarm Optimization (PSO) [3] or adaptive autoscaling [4], [5] have reduced energy costs through consolidation or prediction, yet they often incur high computational complexity and lack adaptability at scale. In contrast, we leverage physics-informed generative modeling to create synthetic, thermodynamically consistent data that empowers AI schedulers to learn optimal energy–performance trade-offs without exhaustive experimentation. Domain-specific strategies, including MapReduceaware scheduling [6], GPU-aware placement using the energy-delay product (EDP) [7], and DVFS-based offline scheduling [8], demonstrate effectiveness in narrow contexts but fail to generalize across diverse architectures. As HPC infrastructures continue to scale toward exascale and beyond, sustainability becomes a fundamental constraint—both in terms of energy efficiency and cooling overhead. Any viable scheduling approach must therefore op-
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
Fig. 1. Power consumption of a typical computing system components
2
We evaluate the CFD-PIVAE system across five representative scientific workflows—Event Reconstruction (WF1), Particle Trajectory Identification (WF-2), Collision Point Detection (WF-3), Pattern Recognition (WF-4), and Anomaly Detection (WF-5)—using multiple schedulers, including First-Come, First-Served (FCFS), Locality-Aware Scheduling (LAS), LAS with Prefetching (LASP), Speculative-Aware Scheduling (SAS) [12], LYNX, and the recent OM-FNN approach. Results show that CFD-PIVAE guided scheduling achieves up to 10% energy savings with only a 5-6% increase in TAT, outperforming conventional methods. Importantly, while LYNX achieves TAT reductions at the cost of higher energy consumption and OM-FNN achieves less energy consumption at the cost of higher TAT, our approach consistently delivers balanced trade-offs. The contributions of this paper are as follows:
erate effectively across large, heterogeneous environments while maintaining thermal and operational stability.
•
•
Despite these advances, three fundamental gaps persist in energy-aware scheduling: 1)
2)
3)
Thermal feedback is ignored. Most existing methods model energy consumption only in terms of CPU frequency or utilization, neglecting the role of heat dissipation, airflow, and cooling inefficiencies. In HPC systems, thermal hotspots directly affect processor reliability and power draw, yet scheduling decisions remain thermally blind. Synthetic data lacks physical realism. Generative models such as Variational Autoencoders (VAEs) [9], [10] have been applied to overcome data scarcity, but standard VAEs produce synthetic workload traces that violate physical constraints. Without grounding in thermodynamics, these datasets lead to scheduling policies that are infeasible under realworld conditions. Over-specialization to hardware. Several approaches target specific ecosystems e.g., Hadoop clusters [6], GPU-centric schedulers [7], or CPUGPU power capping [11]. While effective within their domains, such methods lack generality and are difficult to extend across heterogeneous infrastructures.
These gaps create a misalignment between theoretical scheduling models and practical HPC deployment, limiting the impact of current energy-aware strategies. In particular, the absence of thermodynamically grounded data hinders the design of adaptive schedulers capable of balancing energy efficiency with performance. To address these challenges, we propose a Physics Informed Variational Autoencoder (PIVAE) coupled with Computational Fluid Dynamics (CFD) modeling. CFD provides finegrained simulations of heat propagation, cooling efficiency, and dynamic power dissipation, while the PIVAE ensures that the synthetic data generated for scheduler training respects thermodynamic constraints. This integration enables the adaptive exploration of operational scenarios that are both statistically valid and physically realistic.
•
A novel CFD-informed PIVAE architecture that generates thermodynamically realistic synthetic datasets for energy-aware scheduling. A workflow-specific optimization system that balances CPU utilization, memory access, and thermal constraints to reduce energy consumption without degrading Turnaround time. Empirical validation on a Proxmox-managed HPC cluster, demonstrating scalable energy savings compared with FCFS, LAS, SAS, LYNX, and OM-FNN.
The remainder of this paper is organized as follows: Section 2 reviews related work on energy-aware scheduling. Section 3 presents the proposed CFD-PIVAE architecture. Section 4 details the experimental setup and workflow characterization. Section 5 discusses results across multiple schedulers, and Section 6 concludes with future directions.
2
L ITERATURE R EVIEW
Energy aware scheduling in distributed computing has advanced from early fairness-oriented methods like Round Robin and FCFS [2] to optimization-driven approaches such as PSO-based virtual machines placement [3], which improved energy use but suffered from scalability issues. Machine learning further refined scheduling: Gupta et al. [13] employed VAEs to model energy patterns, while Iqbal et al. [4] introduced adaptive multi-cloud algorithms. However, these often overlooked hardware constraints such as CPU-GPU interactions and thermal dynamics [8]. Zahir et al. [12] extended scheduling with locality awareness, prefetching, and speculative execution, but without addressing energy footprints. Xie et al. [14] present energyefficient scheduling algorithms based on DVFS for real-time parallel applications on heterogeneous distributed systems, achieving substantial energy savings under deadline constraints; however, their approach relies on static analytical models and does not account for thermal dynamics or uncertainty in workload behavior. In contrast, our work addresses these limitations by integrating CFD-informed thermal constraints and probabilistic modeling via PI-VAE to enable energy-aware scheduling under realistic, datadriven, and uncertainty-aware operating conditions. Recent studies have explored predictive and modelingbased methods. Saxena et al. [5] proposed multi-resource
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
prediction for autoscaling, Aldossary et al. [15] used ARIMA to trade off cost and energy, and Dashti et al. [3] applied PSO for consolidation. Other domain-specific solutions include MapReduce profiling [6], EDP-based GPU job placement [7], DVFS task scheduling [8], and ML-driven CPUGPU co-scheduling with power capping [11]. Swarm intelligence methods have also been reviewed for industrial contexts [16]. However, despite these advances, most existing approaches remain limited by their reliance on statistical or heuristic models that ignore underlying physical processes such as heat transfer and cooling dynamics. As a result, their predictions often lack thermodynamic realism and fail to generalize across heterogeneous HPC environments, motivating the need for a physics-informed, scalable framework such as the proposed CFD-PIVAE system. At the infrastructure level, virtualization and consolidation remain central [17], [18], and learning-centric cloud designs reduce carbon footprints [2], but most approaches neglect dynamic workload and thermal variability. Few works integrate thermal feedback into scheduling, despite its impact on reliability and energy efficiency. CFD offers finegrained modeling of heat transfer, yet lacks adaptability. To address this, Physics-Informed VAEs (PIVAEs) combine CFD with generative modeling to create thermodynamically consistent synthetic data, enabling realistic exploration of scheduling trade-offs between turnaround time and energy consumption. Synthetic data generation has been applied using GANs, Monte Carlo methods [19], ARMs [20], RBMs [21], and VAEs [10]. While VAEs stabilize training and capture uncertainty, most models lack thermodynamic realism. Integrating physics into generative systems [5], [7] provides better domain fidelity. However, gaps remain: reliance on simplified energy models [6], hardware-specific constraints [8], and synthetic data divorced from physical laws [15]. This motivates our proposed CFD-guided PIVAE approach, which enforces thermodynamic plausibility while supporting adaptive, energy-efficient scheduling.
3
P ROPOSED S OLUTION
We propose a physics-informed energy-aware scheduling system for High Performance Computing (HPC) workflows that integrates Computational Fluid Dynamics (CFD) with a Physics Informed Variational Autoencoder (PIVAE). Unlike conventional data-driven methods that rely solely on statistical modeling, this system embeds physical constraints such as thermal dynamics and power dissipation directly into scheduling decisions, ensuring that generated operational configurations remain both realistic and deployable in practice. The system operates in three stages. First, execution traces are collected from representative workflows under varying CPU frequencies and scheduling strategies. These traces include execution time, energy consumption, and resource utilization metrics, forming the training dataset for the PIVAE. Second, CFD simulations are employed to capture heat dissipation, airflow behavior, and temperature distribution across HPC components under different workload scenarios. Instead of treating CFD outputs as postanalysis artifacts, their temperature-aware power profiles
3
are integrated into the PIVAE’s latent space, constraining the generative process so that synthetic data reflects both statistical workload behavior and thermodynamic feasibility. Finally, the trained PIVAE produces synthetic datasets that extend the configuration space beyond what is feasible through brute-force experimentation, enabling the discovery of operational scenarios that balance turnaround time (TAT) and energy consumption while respecting physical limits. To connect these modeling capabilities with real scheduling decisions, the system incorporates a scheduling decision module that evaluates workflow requirements and recommends suitable task allocation strategies. We consider six representative schedulers:First Come First Serve (FCFS), which ensures fairness but often neglects efficiency; Locality-Aware Scheduling (LAS), which prioritizes data placement to reduce I/O delays; LAS with Prefetching (LASP), which anticipates overlapping data access patterns; LYNX, which extends locality optimization through predictive prefetching; Speculative-Aware Scheduling (SAS), which uses predictive modeling to enable speculative task execution; and OM-FNN, a machine learning–based approach that leverages optimized feedforward neural networks to predict energy consumption patterns and guide scheduling choices. Together, these schedulers span heuristic-based, predictive, speculative, and learningdriven philosophies, providing a diverse set of strategies through which the system can generate and evaluate physically consistent scheduling scenarios. By constraining synthetic data generation with CFDderived thermal models and applying it across multiple scheduling paradigms, the proposed system enables adaptive scheduling decisions that account for workload diversity, hardware heterogeneity, and thermodynamic constraints in HPC environments. This design unifies datadriven flexibility with physics-based fidelity, providing a principled pathway for energy-aware scheduling that remains realistic, scalable, and broadly applicable. Figure 2 illustrates the overall architecture to identify optimal operational configurations using PIVAE. The VAE utilizes the time and energy consumption data from various scheduling algorithms as input (real data) to generate similar synthetic data, which aids in determining the sweet spot for operational configurations. 3.1 Energy Consumption by Using Computational Fluid Dynamics We use Computational Fluid Dynamics (CFD) to account for the physical principles of heat transfer, thermal conduction, and cooling efficiency, which directly influence processor energy consumption. The CFD provides temperature and heat dissipation profiles under different workload scenarios. These outputs are then integrated into our mathematical energy model, which combines processor power dissipation, memory and I/O activity, and cooling mechanisms into a unified representation of total energy usage. In this way, CFD contributes to the temperature dependent components of the model, ensuring that energy consumption is estimated under conditions that reflect realistic thermal behavior in HPC systems.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
4
Static power arises from temperature-dependent:
leakage
currents
and
Pstatic = Ileakage (T ) · V,
is (5)
where Ileakage (T ) is the leakage current at a given temperature T , and V is the supply voltage. The temperature T is obtained from CFD simulations of heat transfer within the processor, modeled by the heat equation [22] [23]:
∂T + ρcp v · ∇T = k∇2 T + Q (6) ∂t Here, ρ is the density of the material, which corresponds to the processor and the heat sink. cp is the specific heat capacity, v is the velocity field of airflow (if forced convection is present), k is the thermal conductivity and Q is the heat source derived from Pdynamic . ρcp
Fig. 2. CFD-PIVAE Architecture Diagram
To validate and calibrate the CFD simulations, empirical power and thermal data were collected from the underlying high-performance computing (HPC) cluster during workflow execution. Power measurements were obtained using Intel Running Average Power Limit (RAPL) interfaces for per-core energy consumption and IPMI/BMC sensors for system-level power draw, sampled at 100 ms intervals. Thermal data—including CPU core, memory DIMM, and inlet/outlet temperatures—were continuously monitored via LM-Sensors and logged through a Prometheus-based telemetry stack. These real-time readings were used both to parameterize CFD boundary conditions and to cross-verify simulated temperature and heat-flow profiles, ensuring that the modeled results corresponded closely to observed hardware behavior. 3.1.1 Total Energy Model The total energy consumption, Etotal , during the execution of a workflow over time t, is given by: Z t Etotal = Ptotal (t) dt, (1) 0
3.1.3 Memory Power Consumption Memory power consumption, Pmemory , is influenced by dynamic activity, refresh cycles, and standby leakage. Dynamic power arises from read/write operations:
Pmemory-dynamic = Nreads · Eread + Nwrites · Ewrite ,
where workload intensity determines the number of accesses. DRAM refresh power is modeled as:
Prefresh = frefresh · Erefresh ,
where leakage increases at higher operating temperatures. The total memory power is thus:
Pmemory = Pmemory-dynamic + Prefresh + Pmemory-idle ,
3.1.4 I/O Power Consumption The power consumed by I/O operations, PI/O , includes active and idle components:
PI/O = PI/O-active + PI/O-idle . •
Pprocessor = Pdynamic + Pstatic . •
(3)
Dynamic Power
Pdynamic = C · V 2 · f,
(4)
where C is the effective switching capacitance, V is the supply voltage, and f is the clock frequency. Memory and I/O intensive workflows typically involve reduced processor utilization, resulting in lower Pdynamic . •
Static Power
(11)
Active Power:
PI/O-active = RI/O · EI/O ,
(12)
where RI/O is the data transfer rate, and EI/O is the energy per byte transferred. •
Idle Power
PI/O-idle = Pidle · (ttotal − tactive ).
Dynamic power depends on the processor’s switching activity, supply voltage, and operating frequency:
(10)
capturing workload activity, refresh overhead, and thermal effects in HPC environments.
(2)
3.1.2 Processor Power Consumption The power consumption of the processor, Pprocessor , is the sum of dynamic and static power:
(8)
which grows with temperature and is informed by CFDbased thermal profiles. Idle power stems from leakage currents: Pmemory-idle = Ileakage (T ) · V, (9)
where Ptotal represents the total power consumption of the system, modeled as:
Ptotal = Pprocessor + Pmemory + PI/O + Pcooling .
(7)
(13)
3.1.5 Cooling Power Consumption Cooling power, Pcooling , accounts for the energy required to dissipate the heat generated by the processor and other components. Using CFD-derived heat removal rates, Qremoved , the cooling power is given by:
Pcooling =
Qremoved , ηcooling
where ηcooling is the efficiency of the cooling system.
(14)
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
3.1.6
Final Energy Model
Combining all components, the total energy consumption for memory- and I/O-intensive workflows is expressed as:
Etotal =
Z t Pdynamic + Pstatic + Pmemory-active + Pmemory-idle 0
+PI/O-active + PI/O-idle +
Qremoved dt. ηcooling (15)
This model provides a system for analyzing energy consumption in memory and I/O intensive workflows, accounting for processor dynamics, memory and I/O operations, and thermal effects derived from CFD simulations. The priority of the workflows can be used to determine the order in which the workflows are executed, and hence, the order in which the energy consumption is calculated. When calculating energy consumption for CPU usage in data analysis workflows, the focus is on accurately measuring the energy spent on CPU and I/O operations. Given that CPU energy consumption accounts for a significant proportion, approximately 37%, of the total energy usage as highlighted in earlier sections, this focus enables a precise assessment of energy efficiency across various computational workflows. By employing the calculations for CPU energy consumption (E CPU), alongside considerations for memory (E mem), storage (E storage), and network (E network) energy expenditures, the primary objective was to aggregate the energy consumption for each workflow. The scheduling strategies examined within the research context primarily focus on minimizing turnaround times for computational workflows. However, it’s crucial to recognize that these strategies have varying impacts on resource utilization and energy consumption. A scheduling algorithm that excels in reducing turnaround time does not automatically equate to being energy efficient. For instance, efficient scheduling algorithms like speculation or locality aware, while adept at cutting down on turnaround time, tend to be more demanding in terms of computational resources. This higher computational demand invariably leads to increased energy consumption, making these processes more power intensive. In essence, the pursuit of minimized turnaround times through these scheduling strategies must be balanced with considerations for their energy and resource implications, highlighting the complex trade offs involved in optimizing computing workflows. 3.2
VAEs for Energy Consumption Modeling
While real-world workflow traces and energy measurements are inherently limited in scope, this challenge directly motivates our approach. The proposed CFD-PIVAE framework addresses data scarcity by generating physically realistic synthetic samples that emulate diverse operational scenarios. This enables the exploration of energy–performance trade-offs beyond what is feasible through conventional empirical experimentation. We have used a Variational Autoencoder (VAE) to generate synthetic data that represents the connection between energy usage and turnaround time. This data helps predict the best configurations that balances
5
energy efficiency and computational performance in the best possible manner. The dataset contained important parameters such as the number of workflows (N), input data requirements (IR), system usage, turnaround time (TAT), and energy consumption (E) linked to each workflow. Standardizing the dataset ensured consistency among features. We then created the VAE structure by defining the encoder and decoder networks, and selecting the optimal number of layers and activation functions. The loss function, which includes reconstruction loss, was established to direct model optimisation. The VAE was trained by dividing the dataset into training and validation sets. Backpropagation and gradient descent optimisation algorithms were used to iteratively update model parameters in order to minimise the total loss. The optimal configuration for the VAE was determined by tuning hyperparameters and evaluating on the validation set. After training, the VAE model made synthetic data from the learned latent space that showed different operational situations and how they affected the turnaround time. The synthesised data helped analyse the complex connections among operational configuration, turnaround time, and energy consumption in the analysis workflows. 3.2.1
Data Preprocessing and Encoder Design
This section describes the transformation of raw scheduling and energy data into latent representations used by the proposed CFD-PIVAE framework. The process integrates data preprocessing with a physics-informed encoder to ensure that both statistical and thermodynamic characteristics are preserved. Data Preparation: Let X be the input dataset containing the following features: • • • • • • • •
Workflow (WF) Number of tasks (TASKS) Processing time per task CPU frequency usage Turnaround time (TAT) in milliseconds Power consumption by CPU Energy consumption in Joules Energy consumption in kWh
Given a dataset D consisting of tuples (xi , yi , ni , wi , ti , pi , ei , ki ), where xi represents the scheduling technique, yi the workflow, ni the number of tasks, ti the processing time per task, wi the CPU frequency usage, pi the CPU power consumption, ei the energy consumption in Joules, and ki the energy consumption in kWh for each workflow instance i. Preprocessing Steps: 1)
2)
Categorical Encoding: Categorical variables xi and yi are one-shot encoded. This mapping f : C → Rn produces a binary vector f (c) for each category c ∈ C , where n is the number of unique categories. Numerical Feature Scaling: Continuous variables ni , wi , ti , pi , ei , ki are scaled to [0, 1] using:
g(v) =
v − min(v) max(v) − min(v)
(16)
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
3)
4)
6
where v represents all instances of a given feature in D. Data Integration: The encoded categorical and scaled numerical features are concatenated to form the input matrix X ∈ Rm×k , where m is the number of samples and k the total number of processed features. Missing values are imputed via mean substitution. Dataset Splitting: X is partitioned into training (Xtrain ) and validation (Xval ) sets.
where x̂ denotes the reconstructed feature vector derived from the latent variable z . Loss Function: To ensure physical validity and stable latent representations, the total loss integrates three complementary objectives: •
Lrec = ||x − x̂||2 •
These preprocessing steps ensure that scheduling configurations, CPU frequencies, and power metrics are consistently normalized for input to the encoder network. Encoder Design: The encoder component in the proposed CFD-PIVAE differs from conventional VAEs by incorporating both physics-informed and scheduler-aware features. In addition to conventional numerical parameters (e.g., turnaround time, energy consumption, CPU frequency), the encoder integrates thermodynamic indicators obtained from CFD simulations—such as temperature gradients and heat dissipation coefficients—alongside categorical scheduler identifiers (e.g., FCFS, LAS, LASP, SAS, LYNX, OM-FNN). This composite representation enables the model to learn latent embeddings that capture statistical relationships as well as physical dependencies in workflow energy behavior. The encoder computes the mean and variance of the latent distribution as follows:
µ(x), log σ 2 (x) = EncoderNetwork(x) 1 z = µ(x) + exp log σ 2 (x) · ϵ, ϵ ∼ N (0, I) 2
(17) (18)
where x denotes the input vector after preprocessing, µ(x) and σ(x) represent the parameters of the latent distribution, and z is the sampled latent variable. During training, the encoder aligns latent representations with CFD-derived thermal distributions, ensuring that encoded features maintain thermodynamic realism. This allows the VAE to generate synthetic workload configurations that are both statistically valid and physically consistent—enabling data-driven exploration of energy–performance trade-offs across diverse scheduling strategies. 3.2.2 Decoder, Loss Function, and Validation Process The decoder and loss formulation together define how the CFD-PIVAE reconstructs realistic energy–performance profiles while enforcing thermodynamic consistency. After the encoder produces a latent variable z , the decoder reconstructs the input features and estimates corresponding energy and turnaround-time relationships. Decoder: The decoder network reconstructs the input data from the latent representation by minimizing the difference between the original and generated samples while respecting energy-efficiency constraints. This reconstruction enables the exploration of alternative operational configurations that balance energy consumption and performance across various workflows and scheduling techniques:
x̂ = DecoderNetwork(z)
(19)
Reconstruction Loss (Lrec ): Preserves operational characteristics of the original data.
•
(20)
Kullback–Leibler Divergence (LKL ): Regularizes the latent space for smooth and continuous sampling. LKL = DKL q(z|x) || p(z) (21) Here, q(z|x) represents the encoder’s learned posterior distribution over latent variables, and p(z) denotes the prior distribution, typically modeled as a standard normal. CFD-Based Energy Constraint (LCF D ): Adds a physics-informed regularization term derived from CFD-simulated power dissipation.
LCF D =
N X
ECF D (xi ) − Ê(xi )
(22)
i=1
Here, ECF D (xi ) is the energy predicted by CFD simulations for configuration xi , and Ê(xi ) is the corresponding VAE-estimated energy. The total composite loss is formulated as:
L = Lrec + βLKL + γLCF D
(23)
where β and γ control the trade-off between latent-space regularization and CFD-based physical realism. Validation Process: After training, the generated synthetic data undergoes multi-stage validation to ensure thermodynamic plausibility and operational feasibility: •
Data Generation:
•
Y ′ = { yi ∈ Y | D(yi ) = True } (24) Y ′ represents samples that satisfy CFD-based energy constraints. Constraint Verification:
•
V (Y, F ) = { yi ∈ Y | ∀f ∈ F, T (yi , f ) = True } (25) Each generated sample must meet predefined power and thermal thresholds derived from real-system measurements. Cross-Validation and Sensitivity Analysis:
Y = V AE(Data),
C(Y, S) = { (yi , sj ) | yi ∈ V (Y, F ), sj ∈ S }
•
(26)
Assesses the robustness of validated samples across multiple scheduling algorithms. Outlier Rejection: ( True, if yi satisfies CFD-validated constraints f T (yi , f ) = False, otherwise (27) D(yi ) = ExpertReview(yi ) (28)
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
•
Eliminates unrealistic or thermodynamically infeasible samples. Acceptance Criteria:
Acceptable(yi ) = V (yi ) ∧ C(yi , S) ∧ ¬D(yi ) (29) A data point yi is accepted if it meets CFD constraints, passes cross-validation, and is not flagged as an outlier. Integrating CFD-based energy constraints within the VAE’s objective function ensures that synthetic samples adhere to real-world power–temperature dynamics while maintaining diversity. This physically grounded validation process enhances the reliability of generated workload configurations, enabling energy-efficient workflow optimization without sacrificing performance in high-performance computing environments. Algorithms 1 explains the proposed scheduling system, which employs a Physics-Informed Variational Autoencoder (PI-VAE) to generate thermodynamically valid synthetic workload configurations for energy-aware optimization. The model is trained on a real dataset D, which includes workflow execution traces with features such as CPU frequency, turnaround time (TAT), energy consumption, and scheduler identifiers. The VAE consists of an encoder network qϕ (z|x) that maps input data x into a latent space z , and a decoder pθ (x|z) that reconstructs input-like samples from the latent variables. The training process minimizes a composite loss function comprising three terms: (1) the reconstruction loss, which ensures fidelity to the original data; (2) the Kullback–Leibler (KL) divergence, which regularizes the latent space to follow a standard normal distribution; and (3) a physics-informed penalty term, LCFD , that incorporates thermal constraints derived from Computational Fluid Dynamics (CFD) simulations. During training, each mini-batch is encoded into latent parameters (µ, σ), from which latent vectors z are sampled using the reparameterization trick. The decoder then reconstructs each sample x̂, and all three loss components are computed and backpropagated to update the encoder and decoder weights. The CFD penalty is evaluated by simulating the decoded configurations and penalizing outputs that exceed predefined power-temperature thresholds, ensuring thermodynamic realism in the learned representations. Once the PI-VAE is trained, it is used to generate a synthetic dataset S by sampling new latent vectors z and decoding them into configuration samples x̂. Each synthetic sample is evaluated against the CFD constraints, and only those satisfying thermal feasibility are retained. The accepted synthetic configurations are then used to simulate workflow executions under various scheduling strategies (e.g., FCFS, LAS, LASP, SAS). For each configurationscheduler pair, the system computes energy consumption and TAT, forming a set of performance points in a multiobjective space. Finally, efficiency criteria are applied to identify optimal scheduling decisions that provide the best trade-offs between energy usage and execution time. This process enables a principled, data-efficient, and physicsconstrained exploration of scheduling options, bridging machine learning with domain-specific energy modeling.
7
Algorithm 1 Synthetic Data Generation for Energy-Aware Scheduling Using a CFD-PIVAE Require: D: Real dataset of workflow traces {xi } with features: CPU frequency, energy consumption, turnaround time, scheduler ID Require: C : CFD-derived thermal constraints (powertemperature mappings) Ensure: S : Synthetic dataset of thermodynamically valid workflow configurations 1: Define VAE architecture: encoder qϕ (z|x), decoder pθ (x|z) with latent dimension d 2: Define composite loss function: L = Lrecon + βLKL + γLCFD 3: for each mini-batch {xi } ⊂ D do 4: Encode input: µ, σ = Encoder(xi ) 5: Sample latent vector: zi ∼ N (µ, σ 2 ) using reparameterization trick 6: Decode latent vector: x̂i = Decoder(zi ) 7: Compute reconstruction loss: Lrecon = ∥xi − x̂i ∥2 8: Compute KL divergence: LKL = DKL (qϕ (z|xi ) ∥ N (0, I)) 9: Apply CFD constraint: evaluate x̂i via thermal simulation, compute LCFD as penalty if thermal thresholds violated 10: Backpropagate total loss L and update ϕ, θ 11: end for 12: Initialize empty synthetic dataset S ← {} 13: for each sample zj ∼ N (0, I) do 14: Generate configuration x̂j = Decoder(zj ) 15: Evaluate CFD consistency using C : if x̂j violates thermal limits, discard 16: Add accepted x̂j to S 17: end for 18: Use S to simulate scheduling outcomes under multiple strategies (FCFS, LAS, LASP, SAS) 19: Compute energy and TAT trade-offs for each configuration 20: Identify optimal scheduling policy per workflow using efficiency criteria
3.2.3
Error Handling
To avoid the inclusion of unrealistic configurations in the synthetic data generated by the Variational Autoencoder (VAE), we restricted the model to specific operational scenarios. These scenarios included adjustments of 5%, 10%, 15 and 20% reductions in CPU operational configurations. This constraint was imposed to prevent extreme modifications, such as reductions of only 1% or over 50%which could severely disrupt the system’s overall performance. Extreme decreases in operational configurations, particularly those less than 5%, show minimal impact on both energy savings and turnaround time reduction. Conversely, reductions exceeding 20% enhance energy efficiency but at the cost of significantly prolonging turnaround times, thus negatively affecting system performance. This targeted ensures that the synthetic data remains realistic and applicable for optimizing energy efficient scheduling within computational workflows. Given the operational configurations C =
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
8
{5%, 10%, 15%, 20%}, we define the model for generating synthetic data via a Variational Autoencoder (VAE) and its validation through predefined acceptable operational limits. Algorithm 2 Error Handling for PI-VAE Generated Configurations Require: Set of operational reductions X = {5%, 10%, 15%, 20%} Require: Thermal constraint function C(x) from CFD model Ensure: Synthetic data that reflects thermodynamically feasible system performance 1: x ← Execute Algorithm 1 ▷ Obtain real configuration samples 2: for each x ∈ X do 3: Y (x) ← PI-VAE(x) ▷ Generate synthetic data for configuration x 4: Evaluate f (x) ▷ Measure predicted performance (e.g., energy, TAT) 5: if ¬C(Y (x)) then 6: continue ▷ Reject configuration violating thermal limits 7: end if 8: if x < 5% or x > 20% then 9: continue ▷ Skip out-of-scope configurations 10: end if 11: record Y (x) and f (x) ▷ Store valid synthetic outputs 12: end for 13: return All valid Y (x) and associated evaluations f (x) In algorithm 2, the function PIVAE(x) used the generated synthetic data S in algorithm 1 based on input configuration x. The feasibility of each generated configuration is evaluated by the function D(x), which determines if a configuration meets the system’s operational criteria as shown in equation 30, where X represents the input feature space comprising all workflow–scheduler configurations used for model training.
Y (x) = PIVAE(x), ∀x ∈ X ( True if 5% ≤ x ≤ 20% D(x) = False otherwise
(30) (31)
The error handling is modeled as a constraint on the acceptable configurations. The synthetic data must not only simulate realistic scenarios but also conform to specified performance criteria related to energy efficiency and turnaround time (TAT). The acceptable configurations are defined by: ( True if D(xi ) = True Acceptable(yi ) = (32) False otherwise Where yi is a data instance generated from configuration
xi . The objective is to maximize the utility of synthetic data S while ensuring that all data instances yi ∈ S adhere to realistic and permissible operational configurations as defined by D(x).
3.3 Validation of Synthetic Data and Uncertainty Quantification Various studies have utilized bootstrap resampling for uncertainty quantification due to its effectiveness in estimating the variability of statistical estimates [24] [25]. We employ a bootstrap resampling approach on the difference in means between real measurements and PIVAE-generated synthetic data. This gives a non-parametric estimate of variability without assuming any particular distribution of the underlying data, making it well-suited for heterogeneous HPC workloads. Given two samples, Xreal and XPIVAE , representing metrics such as total energy consumption or turnaround time (TAT), the difference in means is defined as:
∆ = X̄real − X̄PIVAE .
(33)
Bootstrap resampling, as shown in algorithm 3, is performed by repeatedly drawing samples with replacement from Xreal and XPIVAE to generate new empirical distributions of X̄real and X̄PIVAE . For each iteration, we compute ∆b , yielding a bootstrap distribution {∆1 , ∆2 , ..., ∆B } over B trials. This allows estimation of a confidence interval (CI) for ∆: CI1−α = Qα/2 (∆), Q1−α/2 (∆) , (34) where Qp denotes the p-th quantile of the bootstrap distribution. The bootstrap distribution also supports hypothesis testing. Specifically, a two-sided p-value for the null hypothesis H0 : ∆ = 0 is estimated from the fraction of resampled statistics on either side of zero. This enables us to assess whether the observed differences between real and synthetic (CFD-informed PIVAE) results are statistically significant. In the context of our system, this procedure provides a principled way to quantify the reliability of PIVAEgenerated energy and performance estimates. By applying bootstrap resampling to metrics across different workflows and schedulers, we capture uncertainty due to limited samples and workload variability, ensuring that the synthetic data maintains not only thermodynamic consistency but also statistical credibility. 3.4 Speculative Scheduling: Time and Energy Sweet Spot To select an optimal operational configuration and scheduling technique for a specific workflow that minimizes energy consumption and TAT, we define an objective function O(F, S) that balances energy E and turnaround time T for operational configuration F and scheduling technique S . Given a synthetic dataset from PIVAE, we apply:
min O(F, S) = αE(F, S) + βT (F, S) F,S
(35)
where α and β are weighting factors balancing the importance of energy savings versus TAT. Algorithm 4 serves as the decision-making core of the proposed CFD-PIVAE system, linking physics-informed modeling with scheduling. For each workflow, multiple schedulers (FCFS, LAS, LASP, LYNX, SAS, OM-FNN) are applied, while processor frequency is adjusted in steps
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
9
Algorithm 3 Uncertainty Quantification using Bootstrap Algorithm 4 optimisation of Energy Consumption for WorkResampling flows Require: Data: Set of workflows 1: Input: Real data D , PIVAE-generated data G, number of {W F1 , W F2 , W F3 , W F4 , W F5 } bootstrap samples B , confidence level CL Require: SchedulingT echniques: 2: Output: Confidence interval for the difference in means {F CF S, LAS, LASP, LY N X, SAS} between D and G Require: F requencyAdjustments: {5%, 10%, 15%, 20%} 3: Normalize the real data D and generated data G 1: Initialize evaluation metrics: Total Energy Consumption 4: Compute mean and standard deviation of D: ( E ), Turnaround Time (T AT ) total mean D, std D D−mean D 2: for each W F in Data do 5: Normalize D: Rnormalized = std D G−mean D 3: for each schedT ech in SchedulingT echniques do 6: Normalize G: Gnormalized = std D 4: for each f reqAdj in F requencyAdjustments do 7: Initialize an array to store bootstrap differences: 5: Apply schedT ech to schedule workflows in bootstrap diffs[B] WF 8: for each bootstrap sample i from 1 to B do 6: Adjust operational configuration according to 9: Resample with replacement from Rnormalized : f reqAdj bootstrap D = sample with replacement(Dnormalized ) 7: Calculate T AT for W F with current 10: Resample with replacement from Gnormalized : schedT ech and f reqAdj bootstrap G = sample with replacement(Gnormalized ) 8: Measure CPU P owerU sage during W F exe11: Compute the mean of the bootstrap samples: cution mean bootstrap D = mean(bootstrap D), 9: Compute Etotal using P owerU sage and T AT mean bootstrap G = mean(bootstrap G) Store Etotal and T AT for analysis 12: Compute the difference in means: diff = 10: 11: end for mean bootstrap D − mean bootstrap G 12: end for 13: Store the difference in bootstrap diffs[i] 13: end for 14: end for 14: Analyze the impact of F requencyAdjustments on 15: Compute the percentiles for the confidence interval: E and T AT across all W F s total 16: lower percentile = (1 − CL)/2 × 100 15: Determine optimal f reqAdj for each W F and 17: upper percentile = (1 + CL)/2 × 100 schedT ech combination minimizing Etotal while main18: Compute the confidence interval for the difference in taining acceptable T AT means: 19: lower bound = percentile(bootstrap diffs, lower %ile) 20: upper bound = percentile(bootstrap diffs, upper %ile) 21: Output the confidence interval: cost becomes O((N/B) · L · d), which scales linearly with data size. [lower bound, upper bound] Overall Complexity:
O(N · F ) + O((N/B) · L · d) ≈ O(N ) (5%, 10%, 15%, 20%) to emulate Dynamic Voltage and Frequency Scaling (DVFS). Turnaround time (TAT) and total energy consumption (Etotal ) are then evaluated, where CFDderived thermal profiles provide temperature-dependent power dissipation and PIVAE generated synthetic data extend the range of workload scenarios. Iterating across all workflow scheduler frequency combinations produces an energy performance landscape from which optimal operating points are identified, ensuring energy efficiency without unacceptable increases in TAT.
3.5
Computational Cost of CFD-PIVAE
The computational cost of the proposed CFD-PIVAE framework consists of two stages: CFD-based data generation and VAE training. Let N be the number of workflow instances, F the CFD feature dimension, L the number of neural layers, and d the latent-space size. CFD Simulation: Each CFD run has complexity O(N ·F ) but is executed once during data preparation. The resulting thermal parameters are reused throughout model training. VAE Training: Each epoch involves forward and backward passes through L layers, requiring O(N · L · d) operations. With mini-batches of size B , the effective per-epoch
(36)
for fixed F, L, and d. Hence, CFD preprocessing adds negligible amortized overhead, and the framework maintains near-linear scalability.
4
E XPERIMENTAL E NVIRONMENT
This work used the DiRAC Data Intensive service (DIaL) at the University of Leicester, managed by the University of Leicester Research Computing Service on behalf of the STFC DiRAC HPC Facility (www.dirac.ac.uk). The DiRAC service at Leicester was funded by BEIS, UKRI and STFC capital funding and STFC operations grants. DiRAC is part of the UKRI Digital Research Infrastructure. The infrastructure comprised dual Intel Xeon Platinum 8280 processors (56 cores, 112 threads per node), 1.5 TB DDR4 ECC RAM, and 10 TB NVMe SSD storage, managed by a Supermicro H11DSiNT Dual Socket Motherboard with redundant 1400W power supplies. All nodes were interconnected via a 200 Gbps HDR InfiniBand network and shared a 4 PB Lustre parallel file system for high-throughput I/O. Power telemetry was captured at 100 ms intervals using Intel RAPL and IPMI interfaces, while thermal metrics were continuously monitored via LM-Sensors. The software environment included
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
Ubuntu 20.04 LTS for orchestration and CentOS 8 for executing distributed workflow simulations. Resource isolation and fine grained energy profiling were achieved using the Proxmox virtualization platform. To assess the thermal effects of our proposed scheduler, we monitored componentlevel temperatures (CPU cores and memory DIMMs) via IPMI/BMC sensors. Heat maps were generated before and after applying the CFD-PIVAE based optimization, capturing spatial thermal distributions. All scheduling strategies were implemented as modular plug-ins within the DiRAC workflow orchestration layer, using Python bindings over the Slurm API to manage task queues, CPU frequency states, and runtime priorities under identical resource conditions. To evaluate the proposed scheduling system, over 4000 real workflow executions were performed across five representative scientific workflows, under varying CPU frequencies and scheduling strategies. These real traces were used to train the PIVAE model, forming the empirical basis for learning energy performance trade-offs. The synthetic operational scenarios generated by the PIVAE were validated through uncertainty quantification using bootstrap resampling, and the optimal configurations it predicted were re-executed on the same cluster to measure actual energy savings and turnaround time improvements. This real world feedback loop ensures that our model is grounded in physical infrastructure constraints and generalizes beyond theoretical assumptions. Figure 3 illustrates the full system setup. Table 1 categorizes a series of computational workflows, identified as WF-1 through WF-5, by their level of resource utilization in a high performance computing environment. Each workflow contains various High Energy Physics (HEP) data analysis tasks. •
•
•
•
•
WF-1 (Event Reconstruction): Classified as a mid to low-level resource utilizer, this workflow involves reconstructing events from raw data collected during experiments. Due to the varying complexity and computational demands associated with different reconstruction tasks, it falls within a broader classification range in terms of resource consumption. WF-2 (Identifying Particle Trajectories): This workflow requires a moderate level of resources (classified as mid) to track particles through detectors. It’s essential for understanding particle interactions but is less demanding than workflows requiring pattern recognition or anomaly detection. WF-3 (Identifying Collision Points): Also classified as mid-level in resource utilization, this workflow deals with pinpointing the locations where particles collide, which is fundamental in experiments that involve particle accelerators. WF-4 (Pattern Recognition): This is a high-level resource utilization workflow due to the complexity of identifying patterns from large datasets. It is crucial for detecting regularities or structures in the data, which may signify underlying physical phenomena. WF-5 (Anomaly Detection): Similar to pattern recognition, anomaly detection is categorized as a high resource utilization workflow. It focuses on identifying data points that deviate from the expected norm,
10
TABLE 1 Resource Utilisation of Different Workflows [26]
Workflow
Workflow Description
WF-1 WF-2 WF-3 WF-4 WF-5
Event Reconstruction Identifying Particle Trajectories Identifying Collision Points Pattern Recognition Anomaly Detection
Resource Utilization Level Mid/Low Mid Mid High High
which can be indicative of new or rare events that require further investigation. Each workflow was parameterized by 500-900 tasks per instance, as table 2 shows the complete experimental P matrix. Energy consumption was calculated as E = (PCP U · texec ), where PCP U denotes CPU power (W) and texec is task execution time. To address data scarcity, a Physics-Informed Variational Autoencoder (PI-VAE) was trained on around 4000 real workflow executions. The model architecture included 6layer encoder/decoder networks with latent dimensions constrained by ANSYS Fluent CFD simulations of heat dissipation. These CFD-derived thermal profiles were integrated into the VAE’s loss function as regularization terms, enforcing thermodynamic plausibility in synthetic data generation. The total loss function combined reconstruction error (Lrec ), KL divergence (LKL ) and CFD-based energy constraints (LCF D ):
L = Lrec + βLKL + γLCF D , where β and γ balanced latent space regularization and physical fidelity.
Fig. 3. Experimental Environment for Energy Aware Scheduling
5
R ESULTS AND DISCUSSION
The primary objective of the experiments was to evaluate the performance of the proposed CFD-PIVAE framework
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
TABLE 2 Experimental matrix Parameters Scheduling WF Description No of Tasks in WF CPU Real CPU Frequency Utilization TAT in ms CPU Power Consumption Energy Consumption in (KW/H)
Description FCFS, LAS, LASP, LYNX and SAS WF-1, WF-2, WF-3, WF-4 and WF-5 WF-1 = 500, WF-2 = 600, WF-3 =750 , WF-4=800 and WF-5= 900 Operational speed of the CPU: 2.1 GHz 80% of the total CPU during task execution, offering a precise measure of CPU engagement. The total time taken from the start to the completion of all tasks. The total power consumed by the CPU during the execution of tasks The cumulative energy used by the CPU during task processing, expressed in (KW/H)
in optimizing energy consumption and turnaround time (TAT) across heterogeneous HPC workflows. The analysis focused on quantifying the efficiency of various scheduling strategies—FCFS, LAS, LASP, LYNX, and SAS—when applied to five representative scientific workflows (WF1 to WF-5) executed on the DiRAC DIaL system. Each configuration was evaluated using CFD-informed thermal and energy measurements to assess the trade-off between reduced energy consumption and sustained computational performance. The results section begins by comparing these scheduling techniques using real data, highlighting the trade-offs between execution time and energy consumption. This is followed by an evaluation of the proposed ML-based approach, which uses a CFD-PIVAE to generate synthetic workflow data. The impact of these synthetic configurations is analyzed in terms of both energy efficiency and TAT improvement. To validate the effectiveness of the MLgenerated solutions, thermal heatmaps of system components are presented, showing before-and-after states with respect to optimization. The section concludes with a detailed explanation of the error handling mechanism for synthetic data generation and an uncertainty quantification strategy, using latent sampling and bootstrap resampling to assess the reliability and robustness of the generated configurations. 5.1 Comparison of Scheduling Strategies: Turnaround Time and Energy Consumption (Real Data) To ensure a fair comparison, all scheduling strategies were implemented within the same workflow execution framework on the DiRAC Data Intensive at Leicester (DIaL) system. The baseline schedulers—FCFS, LAS, LASP, LYNX, and SAS—were developed as plug-ins in the DiRAC orchestration layer using Python bindings over the Slurm API, allowing unified control of job queues, CPU frequency states, and task-level priorities. Each scheduler executed identical workloads and resource constraints so that any observed variations in turnaround time (TAT) and energy consumption were attributable solely to scheduling logic. The proposed CFD-PIVAE scheduler was integrated into this environment to dynamically adjust task allocation and CPU power caps via the RAPL interface at runtime. Every
11
TABLE 3 Task Scheduling and Energy Calculation Parameters and Metrics Sched technique
Work load
Tasks
TAT (ms)
FCFS FCFS FCFS FCFS FCFS LAS LAS LAS LAS LAS LASP LASP LASP LASP LASP LYNX LYNX LYNX LYNX LYNX OM-FNN OM-FNN OM-FNN OM-FNN OM-FNN SAS SAS SAS SAS SAS
WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5
500 600 750 800 900 500 600 750 800 900 500 600 750 800 900 500 600 750 800 900 500 600 750 800 900 500 600 750 800 900
505.07 620.84 763.67 805.31 927.73 461.64 539.05 669.88 726.16 800.15 397.66 452.30 604.29 630.73 684.67 406.60 462.46 617.87 644.90 700.06 345.00 410.00 530.00 575.00 610.00 317.41 380.89 492.72 531.47 584.62
Power Consump (W) 890.94 1095.16 1347.11 1420.56 1636.52 1473.54 1720.63 2138.25 2317.91 2554.08 2006.72 2282.43 3049.45 3182.83 3455.05 2097.92 2386.17 3188.05 3327.48 3612.08 1830.44 2195.12 2898.11 3090.44 3540.98 2152.96 2583.55 3342.09 3604.95 3965.44
Energy Consump (kW/h) 12.50 18.89 28.58 31.78 42.17 18.90 25.76 39.79 46.76 56.77 22.17 28.68 51.19 55.76 65.71 23.70 30.65 54.72 59.61 70.24 17.50 25.00 42.80 49.50 60.00 18.98 27.33 45.74 53.22 64.40
configuration was executed five times, and the reported results represent mean values with deviations below 3%. Table 3 summarizes the comparative results for turnaround time, CPU power consumption, and total energy usage across all workflows. Among the baseline schedulers, the Speculative-Aware Scheduling (SAS) policy consistently achieved the lowest TAT, with WF-5 completing in 584.6 ms compared to 927.7 ms under FCFS. Although FCFS exhibited the lowest instantaneous power draw (42.2 kWh for WF-5), its longer execution time resulted in inferior overall performance. LAS and LASP improved TAT relative to FCFS (e.g., LASP reduced WF-3 from 763.7 ms to 604.3 ms) but increased total energy consumption due to higher CPU utilization and I/O intensity (e.g., LASP consumed 65.7 kWh versus 42.2 kWh for FCFS). In general, FCFS provides predictable but slower task completion as workload scale increases, reflecting its lack of adaptivity. LAS, LASP, and SAS leverage locality and speculative execution to achieve faster completion times, yet at differing energy costs. SAS demonstrates the most balanced trade-off: aggressive speculation enables significant TAT reduction while containing the energy overhead through efficient CPU frequency scaling guided by CFDPIVAE predictions. A closer inspection of WF-1 illustrates this contrast. Under LAS, tasks completed in approximately 461.6 ms at an average effective CPU frequency of 1.70 GHz (81% utilization), resulting in a power draw of 1473.5 W and an energy cost of 18.90 kWh. SAS reduced the same workload’s
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
12
Fig. 4. Turnaround Time Comparison for All Workflows (Real Data)
Fig. 5. Energy Consumption Comparison for all Workflows (Real Data)
completion time to 317.4 ms by increasing CPU frequency to 2.02 GHz (96% utilization), drawing 2153 W and consuming 18.98 kWh—slightly higher instantaneous power but markedly improved throughput. Turnaround Time Trends: Figure 4 compares TAT across scheduling strategies and workflows. SAS outperformed all others, reducing TAT by up to 37% relative to FCFS (e.g., WF-5: 927.7 ms → 584.6 ms). LASP and LYNX achieved moderate improvements in mid-scale workloads (e.g., WF3) with 20–21% reductions, while LAS yielded modest gains of 13–15%. The widening performance gap for larger workflows indicates that SAS scales more efficiently with workload complexity. Energy Consumption Trends: Figure 5 presents the corresponding energy consumption. FCFS recorded the lowest raw energy usage due to minimal scheduling overhead but suffered from high latency. LASP, LYNX, and SAS improved TAT at varying energy costs—LYNX incurred a 91.5% overhead, LASP 79.1%, and SAS 60.1% compared with FCFS for WF-3. Despite this, SAS delivered the most favorable energy–performance ratio, maintaining moderate energy usage while significantly shortening TAT. This trend underscores that as workflows scale, energy efficiency becomes increasingly sensitive to scheduler design; speculative, prediction-driven methods such as SAS and CFDPIVAE offer the most practical balance for high-throughput HPC environments.
For instance, under FCFS for WF-5, reducing the CPU frequency by 15% increased TAT from 927.7 ms to 981.7 ms (a 5.8% rise), while reducing energy consumption from 42.17 kWh to 38.18 kWh. Similar trends are seen for LAS, where for WF-5, the TAT rose from 800.1 ms to 840.6 ms (5.1%), and energy dropped from 55.7 kWh to 53.4 kWh. Notably, SAS demonstrated the most balanced efficiency, where WF-5 under a 15% CPU reduction yielded only a 5.5% increase in TAT (from 584.6 ms to 617.1 ms), while saving 2 kWh (from 58.7 to 56.5 kWh). Across all techniques, energy savings consistently ranged between 3–6% for 10% CPU reductions and 5–8% for 15% reductions, with minor TAT penalties as shown in the table 4. These results validate that CFD-informed synthetic sampling supports practical, thermodynamically feasible decisions for CPU scaling, offering a viable path for energy-aware scheduling optimization in HPC systems. Operational Configurations Reduction Impact on Turnaround Time: The impact of operational configuration reduction on turnaround time (TAT) was further analyzed under the CFD-PIVAE–guided scheduling framework. Figure 6 illustrates that as operational configurations are reduced, both energy consumption and TAT exhibit a correlated response. For example, in Workflow 5 (WF-5), CFDPIVAE optimized the SAS scheduler by reducing configuration utilization by 20%, lowering energy consumption from 62.38 kWh to 54.65 kWh—a 12.4% saving—while maintaining comparable TAT. These results demonstrate that the proposed model effectively constrains speculative scheduling decisions within thermodynamically feasible limits, achieving tangible energy gains without compromising performance. Operational Configurations Reduction Impact on Heat Signatures To evaluate the thermal impact of the proposed energyaware scheduling system, we monitored component-level temperatures (CPU cores and memory DIMMs) using builtin system management utilities (IPMI/BMC). Heat maps
5.2 Impact of CFD-PIVAE on Turnaround Time and Energy Consumption Table 4 presents a detailed comparison of turnaround time (TAT) and energy consumption for all five scheduling techniques across five workloads (WF-1 to WF-5), using CFD-PIVAE-generated synthetic data. The baseline configuration (real CPU setting) is evaluated alongside 10% and 15% CPU reduction scenarios to examine the trade-offs in performance and efficiency.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
13
TABLE 4 Task Scheduling and Energy Calculation Parameters and Metrics: (CFD-PIVAE Generated Synthetic Data) Sched-uling tech FCFS FCFS FCFS FCFS FCFS LAS LAS LAS LAS LAS LASP LASP LASP LASP LASP LYNX LYNX LYNX LYNX LYNX SAS SAS SAS SAS SAS
Work flow
TAT (ms)
WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5 WF-1 WF-2 WF-3 WF-4 WF-5
505.07 620.84 763.67 805.31 927.73 461.64 539.05 669.88 726.16 800.15 397.66 452.30 604.29 630.73 684.67 406.60 462.46 617.87 644.90 700.06 317.41 380.89 492.72 531.47 584.62
TAT (ms) 15% CPU 535.09 653.11 807.49 850.14 981.74 488.34 568.67 710.03 767.83 840.65 421.36 475.38 635.01 662.48 721.11 429.34 489.98 651.55 682.15 741.68 333.96 402.36 520.23 560.01 617.13
TAT (ms) 10% CPU 526.90 648.69 794.42 840.67 967.84 481.02 562.62 699.61 757.34 836.06 414.59 472.26 631.06 658.77 713.09 423.42 481.24 642.79 672.86 729.08 330.60 396.32 512.98 554.20 609.79
Energy (kW/h) 15% 11.75 16.34 25.97 30.14 38.18 17.80 23.44 34.60 42.20 53.41 20.15 26.91 46.31 49.02 57.56 20.14 27.70 50.89 53.84 61.26 17.75 24.83 40.30 48.17 56.51
Energy (kW/h) 10% 12.29 17.15 26.93 31.18 39.67 18.47 24.40 36.06 44.27 55.70 20.85 28.09 48.30 50.78 59.93 21.05 28.98 53.40 56.42 63.91 18.52 25.78 Fig. 7. Heatmap of Component Temperatures Across Five Workflows 41.92 Before and After Optimisation 50.19 58.71
5.3 Uncertainty Quantification using Bootstrap Resampling
Figure 8 presents the bootstrap uncertainty quantification for the difference in mean energy consumption between real and PIVAE-generated samples. The histogram shows the empirical distribution of B = 10,000 resampled mean differences, with the 95% confidence interval (CI) estimated as [0.715, 3.354] kWh. The observed mean difference is approximately 2.03 kWh, indicating that real energy measurements are consistently higher than the synthetic estimates produced by CFD-PIVAE. The bootstrap p-value of 0.0022 provides strong evidence that this difference is statistically significant.
Fig. 6. CPU Impact on Energy Consumption for WF-5 under Different Scheduling (PIVAE Synthetic Data)
were generated before and after applying the PIVAE-based optimization, with temperature data aggregated and visualized using a gradient scale to capture spatial thermal distributions. As shown in Figure 7, the proposed algorithm consistently reduced thermal output, achieving an average 5–7% decrease in component temperatures. This reduction validates the efficacy of our approach in lowering energy usage and highlights additional benefits in terms of cooling efficiency and system longevity.
Rather than implying a systematic bias, this offset reflects the expected outcome of optimization: the PIVAEgenerated configurations correspond to energy-efficient operating states predicted by the model, whereas the real measurements represent unoptimized, measured workloads. The statistical separation between the two distributions therefore suggests that the model has learned meaningful, physically valid relationships that enable reduction of energy consumption without distorting underlying thermal–performance trends. While a final validation through controlled re-execution of selected workflows under the model-recommended settings would provide further confirmation, the present bootstrap analysis demonstrates that the CFD-PIVAE system captures consistent, thermodynamically plausible improvements. This reinforces the reliability of its synthetic data for evaluating energy–performance trade-offs in HPC scheduling.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
14
processing efficiency can be maintained while achieving noticeable energy savings.
Fig. 8. Uncertainty Quantification: Bootstrap Resampling Difference in means: Energy Consumption
5.4 Speculative Scheduling: Time and Energy Sweet Spot Figure 9 illustrates the relationship between energy consumption and turnaround time (TAT) when operational configurations performance is decreased using various scheduling techniques. When the CPU’s performance is decreased to 20%, 15%, 10%, and 5% of its normal operating frequency, we notice a rise in TAT (turnaround time) for all scheduling techniques. This is to be expected, as there is less computational power available to handle tasks. The decrease in processing capability results in higher TAT, which range from approximately 10.02% to 10.86% when the CPU is reduced by 20%. In contrast, there is a proportional decrease in energy consumption, indicating that the system is utilising less power as a result of the diminished performance. Notably, the reduction in energy usage is more significant than the rise in TAT. For instance, in the case of SAS, when the CPU is decreased by 20%, the energy consumption is reduced by approximately 12.76%, which surpasses the increase in Turnaround Time (TAT). The observed pattern remains consistent among the other scheduling techniques and configuration reductions, albeit with slight variations in the ratios. As we decrease the configuration reduction from 20% to 5%, the TAT (turnaround time) progressively decreases, indicating diminishing returns in terms of energy savings compared to time performance. For example, when the CPU is reduced to 95%, the techniques experience an increase in TAT (turnaround time) ranging from approximately 3.05% to 3.90%. Additionally, there is a decrease in energy consumption ranging from about 4.14% to 4.81%. This suggests that once a certain threshold is reached, the amount of energy saved by decreasing CPU performance results in diminishing returns, while still negatively affecting performance. The findings indicate a trend: a decrease of 15% in operational configuration has a disproportionately smaller effect on the turnaround time (TAT) for processing tasks, but it does significantly decrease energy consumption. The TAT increase across various scheduling techniques is only around 5-6%, which is relatively moderate compared to the energy savings of approximately 9-10%. This suggests that by reducing the CPU usage to 15% of its maximum capacity for similar workflow analysis, a significant portion of its
Fig. 9. Impact of reduced CPU frequency on workflow Turnaround Times under different scheduler configurations.(PIVAE Synthetic Data)
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
Within the context of PIVAE generated synthetic data, achieving a 15% reduction effectively balances the trade off between computational performance and power efficiency. It maintains the majority of the CPU’s computational capacity, allowing tasks to be completed in a reasonable amount of time, while also achieving significant energy savings. Based on the synthetic data–driven experiments conducted in this study, a CPU frequency reduction of approximately 15% was observed to offer the most favorable tradeoff between energy consumption and performance degradation for the evaluated workflows and scheduling algorithms. While this observation is specific to the experimental setup and workload characteristics used here, it suggests that moderate frequency scaling can provide meaningful energy savings in similar high-performance computing scenarios without severely impacting turnaround time.
15
trade-offs for compute-dominated workloads, rather than all possible HPC workflow classes. Future work will extend this framework to incorporate memory and I/O behavior, enabling a more comprehensive representation of energy and performance dynamics across diverse workflow categories. Planned extensions include integrating memory bandwidth and I/O models into the PIVAE architecture, as well as conducting systematic comparative evaluations across CPU-, memory-, and I/O-intensive workloads. These enhancements will further strengthen the proposed framework as a general-purpose, physicsinformed solution for energy-efficient and performanceaware scheduling in next-generation HPC systems.
R EFERENCES [1]
6
C ONCLUSIONS AND F UTURE W ORK
This paper presented a physics-informed, data-driven framework for energy-aware scheduling in HighPerformance Computing (HPC) environments, explicitly targeting the joint optimization of energy consumption and turnaround time (TAT). By integrating Computational Fluid Dynamics (CFD) with a Physics-Informed Variational Autoencoder (PI-VAE), the proposed system embeds thermodynamic constraints directly into synthetic data generation, ensuring that predicted scheduling configurations remain both physically plausible and operationally realistic. Unlike purely heuristic or statistical approaches, this integration enables scheduling decisions to be guided by the coupled effects of compute load, energy dissipation, and thermal behavior. Experimental evaluations across multiple scheduling strategies (FCFS, LAS, LASP, LYNX, SAS, and OM-FNN) demonstrated the effectiveness of the proposed approach in identifying a practical operational sweet spot. Specifically, controlled CPU frequency reductions of approximately 15% achieved energy savings of up to 10%, while incurring only a 5–6% increase in turnaround time. These results highlight that meaningful reductions in energy consumption can be achieved without significantly compromising workflow performance, thereby supporting sustainable HPC operation under realistic performance constraints. Furthermore, bootstrap-based uncertainty quantification validated the statistical consistency of PI-VAE–generated results against real execution data, reinforcing confidence in the robustness and reliability of the proposed framework. The current implementation primarily targets CPUintensive scientific workflows, where energy consumption is strongly correlated with computational demand and thermal effects factors that are effectively captured through CFD-based modeling. While workflow scale is varied through task counts, input sizes, and CPU frequency settings to reflect different execution scenarios, this representation does not yet fully encompass memory-intensive or I/O-intensive workflows, whose energy and performance characteristics are influenced by memory bandwidth, data movement, and network behavior. As a result, the present model focuses on representing and optimizing energy–time
D. Versick, I. Wassmann, and D. Tavangarian, “Power consumption estimation of cpu and peripheral components in virtual machines,” ACM SIGAPP Applied Computing Review, vol. 13, no. 3, pp. 17–25, 2013. [2] R. Buyya, S. Ilager, and P. Arroba, “Energy-efficiency and sustainability in new generation cloud computing: A vision and directions for integrated management of data centre resources and workloads,” Software: Practice and Experience, vol. 54, no. 1, pp. 24– 38, 2024. [3] S. E. Dashti and A. M. Rahmani, “Dynamic vms placement for energy efficiency by pso in cloud computing,” Journal of Experimental & Theoretical Artificial Intelligence, vol. 28, no. 1-2, pp. 97–112, 2016. [4] W. Iqbal, J. L. Berral, A. Erradi, D. Carrera et al., “Adaptive prediction models for data center resources utilization estimation,” IEEE Transactions on Network and Service Management, vol. 16, no. 4, pp. 1681–1693, 2019. [5] D. Saxena and A. K. Singh, “A proactive autoscaling and energyefficient vm allocation framework using online multi-resource neural network for cloud data center,” Neurocomputing, vol. 426, pp. 248–264, 2021. [6] N. Yigitbasi, K. Datta, N. Jain, and T. Willke, “Energy efficient scheduling of mapreduce workloads on heterogeneous clusters,” in Green Computing Middleware on Proceedings of the 2nd International Workshop, 2011, p. 1. [7] J. Zhang, H. Wang, H. Lin, and W.-c. Feng, “Consolidating applications for energy efficiency in heterogeneous computing systems,” in 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing. IEEE, 2013, pp. 399–406. [8] X. Mei, X. Chu, H. Liu, Y.-W. Leung, and Z. Li, “Energy efficient real-time task scheduling on cpu-gpu hybrid clusters,” in IEEE INFOCOM 2017-IEEE Conference on Computer Communications. IEEE, 2017, pp. 1–9. [9] S. Gupta and M. K. Gupta, “Variational autoencoders for synthetic workload generation in cloud computing,” Journal of Cloud Computing, vol. 9, no. 1, pp. 1–15, 2020. [10] L. Pinheiro Cinelli, M. Araújo Marins, E. A. Barros da Silva, and S. Lima Netto, “Variational autoencoder,” in Variational methods for machine learning with applications to deep networks. Springer, 2021, pp. 111–149. [11] V. Venkataswamy, J. Grigsby, A. Grimshaw, and Y. Qi, “Rare: renewable energy aware resource management in datacenters,” in Workshop on Job Scheduling Strategies for Parallel Processing. Springer, 2022, pp. 108–130. [12] A. Zahir, A. Anjum, S. N. Srirama, and R. Buyya, “Sas: Speculative locality aware scheduling for i/o intensive scientific analysis in clouds,” Future Generation Computer Systems, vol. 166, p. 107622, 2025. [13] I. Gupta, M. S. Kumar, and P. K. Jana, “Efficient workflow scheduling algorithm for cloud computing system: a dynamic priority-based approach,” Arabian Journal for Science and Engineering, vol. 43, no. 12, pp. 7945–7960, 2018. [14] G. Xie, G. Zeng, X. Xiao, R. Li, and K. Li, “Energy-efficient scheduling algorithms for real-time parallel applications on heterogeneous distributed embedded systems,” IEEE Transactions on Parallel and Distributed Systems, vol. 28, no. 12, pp. 3426–3442, 2017.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2023
[15] M. Aldossary and K. Djemame, “Performance and energy-based cost prediction of virtual machines auto-scaling in clouds,” in 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 2018, pp. 502–509. [16] K. Gao, Y. Huang, A. Sadollah, and L. Wang, “A review of energyefficient scheduling in intelligent production systems,” Complex & Intelligent Systems, vol. 6, pp. 237–249, 2020. [17] J. Shuja, K. Bilal, S. A. Madani, M. Othman, R. Ranjan, P. Balaji, and S. U. Khan, “Survey of techniques and architectures for designing energy-efficient data centers,” IEEE Systems Journal, vol. 10, no. 2, pp. 507–519, 2014. [18] M. A. Alam et al., “Study and analysis of energy efficient data center for sustainable development of ict.” International Journal of Advanced Research in Computer Science, vol. 8, no. 5, 2017. [19] A. Figueira and B. Vaz, “Survey on synthetic data generation, evaluation methods and gans,” Mathematics, vol. 10, no. 15, p. 2733, 2022. [20] D. Viana, R. Teixeira, J. Baptista, and T. Pinto, “Synthetic data generation models for time series: A literature review,” in 2024 International Conference on Electrical, Computer and Energy Technologies (ICECET. IEEE, 2024, pp. 1–6. [21] V. Mnih, H. Larochelle, and G. E. Hinton, “Conditional restricted boltzmann machines for structured output prediction,” arXiv preprint arXiv:1202.3748, 2012. [22] T. L. Bergman, A. S. Lavine, F. P. Incropera, and D. P. DeWitt, Fundamentals of Heat and Mass Transfer, 7th ed. Hoboken, NJ: John Wiley & Sons, 2011. [23] W. Zhong and H. Meidani, “Pivae: Physics-informed variational auto-encoder for stochastic differential equations,” Computer Methods in Applied Mechanics and Engineering, vol. 403, p. 115664, 2023. [24] G. Palmer, S. Du, A. Politowicz, J. P. Emory, X. Yang, A. Gautam, G. Gupta, Z. Li, R. Jacobs, and D. Morgan, “Calibration after bootstrap for accurate uncertainty quantification in regression models,” npj Computational Materials, vol. 8, no. 1, p. 115, 2022. [25] T. Endo, T. Watanabe, and A. Yamamoto, “Confidence interval estimation by bootstrap method for uncertainty quantification using random sampling method,” Journal of Nuclear Science and Technology, vol. 52, no. 7-8, pp. 993–999, 2015. [26] P. Buncic, M. Krzewicki, and P. Vande Vyvre, “Technical design report for the upgrade of the online-offline computing system,” CERN, Tech. Rep., 2015.
16
Prof. Ashiq Anjum is a Professor of Distributed Systems at the School of Computing and Mathematical Sciences, University of Leicester, where he also serves as the Director of Enterprise and Impact. He is the AI Lead for the £60 million METEOR project at Space Park Leicester. Previously, he was a Professor of Distributed Systems and Director of the Data Science Research Centre at the University of Derby, U.K. His research interests include data-intensive distributed systems, distributed machine learning, self-learning digital twins, and high-performance analytics for streaming data. He has collaborated with Rolls-Royce, BT, and healthcare providers on AI and distributed computing projects. Professor Anjum has an extensive publication record and an H-index of 39. (e-mail: [email protected])
Prof. Mark I. Wilkinson is with the Department of Physics and Astronomy, University of Leicester, United Kingdom. He obtained a B.A. and M.Sc. in Theoretical Physics from Trinity College Dublin and a D.Phil. in Theoretical Astrophysics from the University of Oxford under Professor Wyn Evans. His research focuses on the dynamical modelling of dark matter in galaxies, gravitational lensing, and the application of machine learning to astrophysical problems. Prof. Wilkinson currently serves as Director of the DiRAC High-Performance Computing Facility, leading national efforts in computational astrophysics and large-scale data analysis. (email: [email protected])
AUTHOR B IOGRAPHIES
Dr. Ali Zahir is a Postdoctoral Fellow at the School of Computing and Mathematical Sciences, University of Leicester, United Kingdom. His research focuses on enhancing data processing workflows in cloud and edge computing environments, with emphasis on High-Energy Physics (HEP). He has contributed to advanced data retrieval algorithms and distributed systems, and has collaborated with CERN on the ALICE project since 2009. Beyond academia, Mr. Zahir engages in research initiatives in data science and high-performance computing, bridging the gap between theory and implementation in distributed computing. (e-mail: [email protected])
Dr. Jeyan Thiyagalingam leads the Scientific Machine Learning (SciML) Research Group at the Rutherford Appleton Laboratory, Science and Technology Facilities Council (STFC–UKRI), Harwell, United Kingdom. His group develops and applies machine learning and signal processing techniques to address complex scientific challenges. Prior to joining STFC–RAL, Dr. Thiyagalingam was an Assistant Professor at the University of Liverpool and held research appointments at the University of Oxford, including a James Martin Fellowship. His interests span scientific machine learning, data-driven modelling, and advanced signal processing. He is a Fellow of the British Computer Society (BCS), a Senior Member of the IEEE, and an Associate Editor for Patterns, a Cell Press journal on data science and AI. (e-mail: [email protected])