arXiv:2604.25540v1 [cs.DC] 28 Apr 2026
Economical and ecological impact of sector coupling applied to computing clusters Philip Bechtle1 , Oliver Freyermuth1 , Max Geffers4 , Manual Giffels2 , Michael Hübner1 , Florian Kirfel1,* , Jochen Kreutz3 , Stefan Krieg3 , Sebastian Matberg1 , and Matthias Schnepf2 1
Physikalisches Institut, University of Bonn, Bonn, Germany 3 Karlsruhe Institute of Technology 4 Forschungszentrum Jülich 2 Universität Heidelberg * Contact: [email protected] April 29, 2026
Abstract: The rising share of abundant renewable energy inevitably increases volatility in the electricity production. The concept of sector coupling means that the volatility of electricity production to a large degree can be absorbed by dispatching electricity consumption whenever excess renewable energy is available. A system that is dynamically operated based on this principle can lower its total environmental impact. In addition, operational costs might be reducible as electricity prizes strongly depend on the residual load of the energy system. High-performance computing clusters in the field of science represent an ideal testing ground for such dynamic operation. Short-term delays in computing results due to electricity production being associated with high costs or carbon emissions are often negligible, provided that an overall computing target remains constant over long time periods. This study simulates the simplified operation of computing clusters using publicly available data on electricity production in Germany. The optimal utilisation along with associated carbon emission and cost reductions are determined separately. Hardware acquisition costs and embedded emissions are taken into account. The stability of a fixed computing target given the determined utilisation optima is evaluated in two validation periods. Additional simulations with modified parameters are carried out to estimate potential conditions under which dynamic operation of a computing cluster would continue to enable savings in the future.
1
Section Contents
2
Contents 1 Introduction
2
2 Background
3
3 Methodology
4
4 Simulation results 4.1 Optimisation with respect to carbon emission . . . . . . . . . . . . . . . . 4.1.1 Dynamic operation and clock frequency limitation - comparison . . 4.1.2 Emission operation threshold - validation . . . . . . . . . . . . . . . 4.2 Optimisation with respect to operational costs . . . . . . . . . . . . . . . .
8 8 13 15 15
5 Conclusion
17
1 Introduction The global warming induced by carbon emissions, first predicted in 1896 [1], is expected to cause a global increase in average temperatures of more than 1.5 ◦ C by 2030 compared to pre-industrial levels [2]. The consequences of this trend are already evident across Europe in the form of more frequent and more intense extreme weather events. In Germany alone, the annual costs of recorded extreme-weather-related damages exceeds six billion euros during the period from 2015 to 2024 [3]. To at least mitigate these effects in the future, Germany, originally along with 195 other nations, committed itself to the Paris climate agreement with the goal of limiting global warming to well below 2 ◦ C above pre-industrial levels [4]. This requires a drastic reduction in greenhouse gas emissions, primarily achievable through the switch to renewable energy sources for power generation. In 2016, the share of renewable energies in Germany electricity consumption was already at 31.7 % [5]. However, its share in the final total energy consumption only amounted to 14.6 % [5] and was thus significantly lower. This discrepancy can be attributed to sectors such as transport and heating, both of which still rely heavily on fossil fuels. Naturally, attempting to electrify these areas as well requires more energy to be produced by renewable energy sources, i.e. volatile wind and solar power will become the dominant power sources. Ultimately this increases fluctuations in the energy grid. To control such fluctuations, a strong sector coupling [6] is required, capable of leveraging the flexibility of various sectors. In contrast to large industrial plants, which, to be profitable, must often operate continuously, the focus for sector coupling lies on systems that can be dynamically regulated. An almost perfect use case for this desired dynamic regularisation are computing clusters, as their workload can be flexibly controlled by shifting computing tasks over time. As of 2025 data centres already account for 4 % of Germany’s gross electricity consumption [7] and, given the recent developments in the field of artificial intelligence, will account for an ever-increasing share of consumed power. Forecasts assume that, by 2030, computing might be responsible for up to 8 % of the total power consumption [8]. Shifting computing tasks to periods with an oversupply of renewable energies can help at
Section 2 Background
3
stabilising the energy grid. At the same time, procuring electricity from the spot market offers significant savings in terms of carbon emissions, while also enabling economically optimised operation of computing clusters. As an example, negative wholesale energy prices on the German electricity spot market were reported for a total of 573 hours in 2025 [9], primarily at weekends or on public holidays during which renewable energy sources led to an oversupply. Targeted ramp-up of a computing cluster at such periods therefore offers potential cost savings. This dynamic operation of computing clusters inevitably leads to delays in computing results on short time scales. However, this is often negligible, especially in the field of science, making computing clusters in research institutions and universities the ideal place to test such an application. The following studies are therefore carried out by relying on different computing cluster operated by research facilities. The main focus is placed on the BAF high throughput computing cluster [10], operated at the Physics Institute of the University of Bonn. This cluster is primarily used for computing tasks in the field of high-energy physics, which, with more than 20 % [11], has the greatest demand for computing power in the German scientific sector and thus also represents the largest lever for dynamic control. Initially, in section 2 various options for reducing greenhouse gas emissions in the area of computing clusters are discussed. Particular emphasis is placed on the feasibility of the respective countermeasures on a short time scale. Building on this, section 3 introduces the simulation of a dynamic cluster operation for data centres by relying on public data on Germany’s electricity generation in recent years. Finally, section 4 presents results for concrete case studies based on this simulation.
2 Background In general, computing clusters account for a significant share of total emissions in modern research. In particular, large high-energy physics experiments almost all rely on computing clusters. A study evaluating the professional carbon footprint of doctoral researchers at one of the four major LHC experiments shows that computing with 1.91 tCO2 accounts for 9.3 % of the experiment’s total emissions [12]. Fortunately, computing clusters offer many possibilities to reduce their carbon footprint either directly or indirectly. The first key decision for any newly built computing cluster involves selecting an adequate location. Besides the clearly important access to power supply, the local climate also plays a significant role. Depending on a cluster’s architecture, current outside temperatures in Germany allow operation without active cooling [13]. This leads to a significant reduction of carbon emissions associated with the cluster operation. The heat, inevitably generated during operation, can be fed into the local heat networks of neighbouring towns. However, the temperature gradient between existing heat network’s operating temperatures and the waste heat generated often poses a problem [13]. Regarding a computing cluster’s installed hardware, using more modern components usually goes hand in hand with lower carbon emissions. This is either achieved through the components being more energy efficient or simply due to their increased performance, which shortens the overall runtime of computing tasks. Obviously, the embedded emissions associated with producing and transporting hardware have to be factored in.
Section 3 Methodology
4
Ultimately, training end users also offers a great indirect potential for savings. Every end user initially must learn to correctly estimate the cluster resources required for their own computing tasks, e.g. the number of CPUs needed to efficiently complete a certain task. Furthermore, the correct use of an appropriate programming language is crucial to make optimal use of the available hardware. In large scientific collaborations in particular, further savings can be achieved by adopting the FAIR data principle [14]. This principle intends to ensure that data and software are easily findable, accessible, interoperable and maybe most importantly reusable. Such a reusability of results helps at eliminating redundancies and thus lowers unnecessary emissions associated with computationally intensive tasks. A very comprehensive overview of potential energy and carbon emission savings for computing clusters can be found in Refs. [15] and [13]. In order to operate a computing cluster that complies with increasingly stringent emission regulations in Germany [16], progress is needed across all these aspects. Whether by modifying clusters to utilise waste heat, modernising hardware or providing intensive training for end users, progressing towards a carbon-neutral operation inevitably involves considerable effort and expenses. However, these measures vary significantly in terms of the effort required for implementation. Particularly when targeting short- and medium-term emission reductions, the dynamic operation of a cluster, as introduced in section 1, involves significantly less work than, for example, completely rebuilding a cluster in order to benefit of modern cooling technology and hardware. To implement dynamic operation, a cluster generally needs to be equipped with a control software. Based on publicly available weather data and information on energy generation, this software can automatically decide whether to accept or continue computing tasks at a given time, or whether the cluster should switch to an idle state and resume work when conditions are more favourable. Such a cut-off in terms of carbon emissions per MWh up to which a operation is deemed favourable could offer significant short-term potential in terms of carbon emission and operation cost reduction. In addition, given some flexibility with respect to key parameters, once implemented such a control software could be re-used for different clusters.
3 Methodology The conducted simulation1 is intended to assess the optimal operation of a computing cluster in the field of high-energy physics. Both the minimisation of carbon emissions and reductions with respect to a computing cluster’s operating costs are being targeted. For the sake of simplicity, it is assumed that the cluster is entirely powered by spot market-traded electricity. The data, covering both the cost of consumed electricity in €/MWh as well as the net electricity generation in MWh, was obtained using the available energy data in Germany, publicly provided by the Fraunhofer Institute for Solar Energy Systems ISE2 [17]. Their database collects net electricity production broken down by generation type along with the spot market prices for time intervals of 15 minutes. The corresponding carbon emissions associated with each generation type per year can be accessed separately via Ref. 1
The complete simulation workspace can be found under https://gitlab.uni-bonn.de/grp_phy/ drittmittelprojekte/erum-data/fidium/re-study/-/tree/main/fidium_project. 2 The website can be found under https://energy-charts.info.
5
Section 3 Methodology
CO2 Epower / kgMWh
[18], allowing the carbon emissions to be calculated in kgCO2 /MWh for each 15 minute interval. The resulting price and carbon emission development over a one-year period are shown exemplary for the calendar year 2024 in fig. 1a and fig. 1b.
800 600 400 200
01/01/2024
01/04/2024
01/07/2024
01/10/2024
01/01/2025 Date
01/10/2024
01/01/2025 Date
e Cpower / MWh
(a)
750 500 250 0
01/01/2024
01/04/2024
01/07/2024 (b)
Figure 1: Development of (a) carbon emissions in kgCO2 /MWh and (b) costs in €/MWh arising from German net electricity generation for the calendar year 2024. As already described in section 2, a short-term delay when processing computing tasks is acceptable during time periods where electricity is either expensive or associated with high carbon emissions. Nevertheless, the total available computing power for research should remain approximately constant over a longer period of time. This implies that a cluster which, due to high carbon emissions, can only be operated for 50 % of a given time period must, assuming linear scalability, be designed twice as large. This way a constant compute target can be maintained. For the sake of simplicity, a homogeneous cluster consisting of identical CPUs was simulated in this study. This allows the variable cluster size, depending on the possible cluster utilisation to be modelled directly via the number of logical cores of all combined CPUs. Conversely, the estimated embedded carbon emission for every single
6
Section 3 Methodology
logical core has to factor in all components required to expand the computing clusters. Electricity consumption and thus also the operating emissions for day-to-day operations were calculated based on the computing cluster’s estimated electricity consumption and the aforementioned data regarding public net electricity production in Germany. Similar to the embedded carbon emissions, the total power consumption of the cluster was calculated based on the scaling factor, i.e. the number of logical cores. To allow for a more realistic simulation, different workload scenarios were specified. Each scenario is defined via the respective partial load power consumptions and the relative time fractions during which the computing cluster is operated at each load. This is particularly relevant to avoid neglecting the power consumption of a computing cluster during idle phases. To enable rapid start-up under more favourable conditions and limit hardware damage, complete shutdown was not investigated. Therefore, even in the case of high emissions associated with the net electricity production, i.e. periods in which operation is generally avoided, a non-zero electricity consumption was assumed. Under these assumptions, the total carbon emissions Etotal are calculated through Etotal =
X ncores ncores · Eembedded, CPU hour · ttotal + · Pavg. · (Epower,i · t) u i | u {z } Eembedded
+
|
{z
Eoperation
}
X ncores · Pidle · (Epower,j · t) , u j |
{z
Eidle
}
where ncores denotes the number of logical cores, u the relative utilisation within the considered period ttotal , and t the respective sampling time interval. The carbon emissions of electricity generation in kgCO2 /MWh for each time interval are represented by Epower . The index i runs over all intervals in which Epower is below the utilisation threshold value Xemission . In contrast, the index j covers the time intervals in which the cluster remains in an idle state as Epower exceeds Xemission . The parameters Pavg. and Pidle represent the workload scenario dependent average power consumption per logical core during operation phases and idle phases, respectively. The optimal threshold value for the emissions associated with the net power generation can be found by minimising Etotal as a function of the relative cluster utilisation u. Any time the net power production exceeds this threshold it is more favourable to shut down the computing cluster, i.e. remain in idle mode. A minimum far from u = 1 implies, that the computing cluster must be extended to fulfil the constraint arising from the constant compute target. Similarly, an optimisation was carried out with regard to the financial costs of operating a computing cluster. As the constant compute target was again modelled via the number of logical cores ncores , the price for the purchase of a single core Cacq. must reflect all additional acquisition costs for the cluster expansion. Once this price is defined, the total costs Ctotal
7
Section 3 Methodology can be calculated via ncores ncores Ctotal = · Cacq., CPU hour · ttotal + · Pmax · Cyearly demand · ttotal | u {z } | u {z } Cacq.
Cdemand
X X ncores ncores · Pavg. · · Pidle · + (Cpower,i · t) + (Cpower,j · t) . u u i j |
{z
Coperation
}
|
{z
Cidle
}
Here, Pmax denotes the maximum power consumption per core, which, when multiplied by the number of cores and the electricity provider’s yearly demand charge, constitutes the second term of the total cost. The operating costs can be determined using the price per power produced at each sampling interval Cpower and the average power consumption Pavg. of a defined workload scenario. Electricity costs during idle periods are determined via the idle power consumption Pidle . The used indices are defined analogously to the emissions calculation, i.e., i again covers all intervals in which Cpower lies below a derived threshold value Xcost in €/MWh, while j runs over all intervals exceeding this threshold. The base charge of any electricity contract was disregarded in this model, as it would only add an offset to the overall function. A definition of all parameters used in the simulation can be found in table 1. Table 1: Simulation parameters and their definitions. Parameter
Unit
ncores u Etotal
kgCO2 kgCO2 /h kgCO2 kgCO2 /MWh kgCO2 kgCO2 € €/h € €/kW/year € €/MWh € € MW MW MW h h kgCO2 /MWh €/MWh
Eembedded, CPU hour Eembedded Epower Eoperation Eidle Ctotal Cacq., CPU hour Cacq. Cyearly demand Cdemand Cpower Coperation Cidle Pavg. Pmax Pidle ttotal t Xemission Xcost
Description Default number of logical cores in a computing cluster. Relative operating time of the cluster throughout the simulated period (0 ≤ u ≤ 1). Total carbon emission throughout the simulated period. Hourly embedded emissions per logical core, based on an assumed lifetime of ten years for all components. Total embedded emission throughout the simulated period. Carbon emissions for electricity generated in Germany at each sampled time interval. Total carbon emissions from power consumption during cluster operation throughout the simulated period. Total carbon emissions from power consumption during idling throughout the simulated period. Total cost throughout the simulated period. Hourly acquisition cost per logical core, based on an assumed lifetime of ten years for all components. Total acquisition cost throughout the simulated period. Electricity supplier’s capacity-based charge per year. Total demand cost throughout the simulated period. Spot market energy price for electricity generated in Germany at each sampled time interval. Total cost from power consumption during cluster operation in the simulated period. Total cost from power consumption during the cluster’s idle state in the simulated period. Workload-dependent average power consumption per logical core. Maximum power consumption per logical core. Power consumption per logical core when in idle mode. Total time period used in the simulation. Sampling interval duration. Emission threshold above which the computing cluster remains in idle state. Cost threshold above which the computing cluster remains in idle state.
In order to evaluate the determined emission threshold’s stability the calculated optimum over a long period, the optimal threshold was extrapolated to a test period covering the calender years 2023 and 2025 for specific scenarios. In both years, the extrapolated threshold extra. Xemission was compared to thresholds explicitly derived for each of those validation years val. Xemission . To allow for a realistic comparison, the increase in renewable energies in Germany extra. and its impact on emissions were included in Xemission . Thus, the extrapolation factors in an offset derived via public data, again provided by the Fraunhofer ISE platform [19]. To derive this offset, a linear relationship between the public energy production’s average
Section 4 Simulation results
8
carbon emission per year and the relative share of renewable energies was assumed. Using data covering the years from 2002-2025 an increase of 1 % in the share of renewable energies was found to corresponds to an average emission reduction of 5.4(3) kg.
4 Simulation results The dynamic operation of computing clusters was simulated based on the 2024 data. The optimisation with regard to the total carbon emissions, discussed in section 4.1, was carried out for the following five different computing clusters setups: • BAFdefault : Default worker nodes of the Bonn Analysis Facility (BAF) high throughput cluster installed at the Physics Institute at the University of Bonn. • BAFmodern : Modern worker nodes of the BAF high throughput cluster installed at the Physics Institute at the University of Bonn. • DEEPCM : Part of the DEEP module supercomputer prototype [20] at the Jülich Supercomputing Centre (JSC) [21]. Designed for high single-thread performance and a modest amount of memory. • DEEPDAM : Part of the DEEP module supercomputer prototype at the JSC. Designed for data-intensive analytics and machine learning applications. Accelerators are ignored in the simulation. • GridKaARM : ARM worker nodes in the WLCG’s Tier 1 Grid Computing Centre Karlsruhe (GridKa) [22]. It is important to note, that the tested cluster setups do not exactly correspond to their real-life counterparts. Instead, a simplified setup, neglecting heterogeneous hardware configurations and accelerators was simulated. The optimisation in terms of total operating costs, discussed in section 4.2, was only carried out for the two BAF cluster scenarios.
4.1 Optimisation with respect to carbon emission An overview of the parameters used for emissions optimisation across all the different scenarios can be found in table 2. The first difference between the setups concerns the maximum power consumption per logical core Pmax , which was determined via full load tests. The resulting values naturally depend on the hardware used as well as the specific cluster settings, and vary between 9.2 W (BAFdefault ) and 2.9 W (GridKaARM WN ). The associated idle values Pidle range between 15 % (BAFmodern ) and 48 % (DEEPDAM ) of the respective maximum power consumption. Whilst the power consumption of each individual setup was directly measured, no precise data was available for the embedded emissions of the installed hardware. Therefore, Eembedded, CPU hour in all five setups was estimated using a life cycle assessment of the Dell R740 rack server conducted by the thinkstep AG [23]. The study lists embedded emissions of 4092 kgCO2 per rack server, taking into account manufacturing costs (4288 kgCO2 ), transport costs (3 kgCO2 ) and end of life credits (−199 kgCO2 ). With approximately 80.3 %
9
Section 4 Simulation results
Table 2: Parameter values chosen for the simulation of all five cluster scenarios. Parameter
Setup
ncores Pmax / W Pidle / W Eembedded / kgCO2 /h
BAFdefault
BAFmodern
DEEPCM
DEEPDAM
GridKaARM WN
7104 9.2 2.3 1.5 × 10−5
2816 7.6 1.1 1.8 × 10−4
2400 7.8 2.6 1.8 × 10−4
1536 6.9 3.3 1.8 × 10−4
2816 2.9 0.9 1.8 × 10−4
the dominant share of manufacturing emissions is accounted for by the SSD production. To express the embedded emissions per logical core, it was assumed that 256 logical cores can be installed per rack server. Assuming a life cycle of 10 years, this resulted in Eembedded, CPU hour = 1.8 × 10−4 kgCO2 /h for all but the BAFdefault setup. In contrast to the others, the BAFdefault setup, solely relies on HDDs as storage media, for which significantly lower embedded emissions had to be assumed. As a reference, the study conducted in [24], which compared life cycle assessments from various manufacturers to derive an average storage emission factor SEF of 0.16 kgCO2 /GB for SSDs and 0.02 kgCO2 /GB for HDDs, was used. Applying these storage emission factors, the BAFdefault setup’s embedded emissions amounted to 1.5 × 10−5 kgCO2 /h. It should be noted at this point that the embedded emissions evaluated in Ref. [24] displayed significant fluctuations, especially when comparing different manufacturers. Thus, the exact value of Eembedded is highly dependent on the chosen setup and the installed hardware. All values chosen here are only rough estimations and assumed to be associated with considerable uncertainties. In order to simulate realistic use cases of a computing cluster, different workload scenarios were considered for all setups. Besides a medium workload and a heavy workload scenario, which are oriented towards studies in Ref. [23], a backfilling [25] scenario was defined. Such a scenario is frequently used in high-energy physics and allows free resources to be allocated more efficiently by prioritising short single-core jobs. As a result, the cluster can be assumed to run under full load during the simulated period. The exact parameters of all three scenarios are summarised in table 3. For each load working point, it was assumed that a cluster’s power consumption scales linearly between Pidle and Pmax . Table 3: Workload scenarios used in the simulation. Setup dependent power consumption is assumed to scale linearly between Pidle and Pmax . Load mode / %
0
10
50
100
Medium workload time fraction / % Heavy workload time fraction / % Backfilling workload time fraction / %
25 10 5
30 20 0
35 55 0
10 15 95
The optimisation results of the five setups are summarised in table 4 for all workload scenarios. In addition, Etotal alongside the individual contributions is displayed in fig. 2 as a function of the utilisation u in case of BAFmodern ’s backfilling scenario and illustrates the expected trend: Eoperation decreases along with the utilisation, as the cluster uses electricity
10
Section 4 Simulation results
with increasingly lower levels of associated carbon emissions. At the same time, Eidle and Eembedded along with the number of logical cores, i.e. the cluster scaling factor, increase to maintain a constant compute target. Table 4: Optimisation results to minimise carbon emissions for all setups and workload scenarios. Listed is the optimal utilisation uopt. , the associated threshold Xemission above which a computing cluster switches to idle state, and the relative emissions compared to constant operation Etotal (uopt. )/Etotal (u=1). In case that a constant operation yields the lowest total carbon emissions, no threshold on Epower is specified. Workload scenario
uopt.
Xemission / kgCO2 /MWh
Etotal (uopt. )/E
total (u=1)
BAFdefault Medium Heavy Backfilling
1.000 0.975 0.745
699.90 515.01
1.000 0.998 0.955
BAFmodern Medium Heavy Backfilling
0.983 0.882 0.635
714.13 607.45 458.11
0.999 0.988 0.918
DEEPCM Medium Heavy Backfilling
1.000 1.000 0.916
635.90
1.000 1.000 0.993
DEEPDAM Medium Heavy Backfilling
1.000 1.000 1.000
-
1.000 1.000 1.000
GridKaARM Medium Heavy Backfilling
1.000 1.000 0.953
671.458
1.000 1.000 0.997
A comparison across the five setups generally demonstrates that emission reduction through dynamic operation is increased for workloads with high cluster utilisation. In addition, it is apparent that the emission reduction strongly depends on the ratio between Pidle and Pmax . If power consumption is not reduced significantly when idling, the benefits of a dynamic computing cluster operation are negated. Consequently, the greatest potential for savings is found for the BAFmodern setup when simulated for the backfilling workload scenario. At a utilisation of u = 0.635, carbon emissions can be reduced by around 8 % compared to u = 1. In contrast, the DEEPDAM setup offers no potential for savings.
E / 105 kg CO2
1.2
Etotal Eoperation
1.0
Eidle
Eembedded 2.5 ncores uopt. = 0.635 2.0
0.8
1.5
0.6
1.0
0.4
0.5
0.2 0.0
ncores /104
11
Section 4 Simulation results
0.2
0.4
0.6
0.0
0.8 u
Figure 2: Display of total emission alongside the individual contributions (left y-axis) and number of logical cores needed to maintain the default compute target (right y-axis) as a function of the utilisation u. The optimal cluster utilisation is marked by the black vertical line. Its relatively high power consumption in idle mode (Pidle/Pmax ≈ 48 %) results in a trivial minimum of u = 1. The actual idle power consumption is not only hardware-dependent but can also be adjusted via BIOS energy settings. To illustrate this variability and assess its effect, fig. 3 shows the relative emission reduction as a function of the relative idle power consumption Pidle/Pmax . For better visualisation, only the backfilling scenario is displayed. The strong dependency of the emission reduction on the ratio is evident. Potential savings compared to a continuous operation decrease rapidly as the relative idle power consumption increases. Beyond a threshold of Pidle/Pmax ≈ 0.4, dynamic operation no longer offers any advantages regardless of the simulated setup. In contrast, assuming a cluster can be completely shut down instead of idling (Pidle/Pmax = 0), savings of more than 50 % could be achieved depending on the setup. To further estimate which effect the uncertainty with regard to embedded emissions has on the overall optimisation, the relative savings were derived as a function of embedded emissions. This test was only performed for the BAFmodern ’s backfilling scenario. All other simulation parameters were kept unchanged. The results for an up- and down-variation of 50 % with respect to the nominal value of 1.8 × 10−4 kgCO2 /h are listed in table 5. The results show the expected trend: increased embedded emissions shift the minimum closer to u = 1 and vice versa. Even for the up-variation, the calculated optimum corresponds to a utilisation smaller than u = 1. The associated relative emission reduction at approximately 5 %. In the tested setup, dynamic operation of a cluster enables a carbon emission reduction across a fairly wide range of Eembedded . This is consistent with the results shown in fig. 2 which already illustrated that the embedded emissions do not represent the optimisation’s
12
Etotal (uopt. ) Etotal (u=1)
Section 4 Simulation results
1.0 0.9 0.8 0.7
Nominal setup values BAFdefault DEEPDAM BAFmodern GridKaARM DEEPCM
0.6 0.5 0.0
0.2
0.4
0.6
0.8
1.0 Pidle Pmax
Figure 3: Relative emissions compared to constant operation Etotal (uopt. )/Etotal (u=1) as a function of the power consumption ratio Pidle/Pmax for all setups in the backfilling scenario. Nominal power consumption ratios are marked.
13
Section 4 Simulation results dominant factor.
Table 5: Optimisation results to minimise carbon emissions for the BAFmodern setup’s backfilling scenarios. Listed is the optimal utilisation uopt. , the associated threshold Xemission beyond which a computing cluster switches to idle state, and the relative emissions compared to constant operation Etotal (uopt. )/Etotal (u=1). In addition to the nominal value, the results are shown for a 50 % up- and down-variation of the embedded emissions. Workload scenario
uopt.
Xemission / kgCO2 /MWh
Etotal (uopt. )/E
total (u=1)
BAFmodern Backfilling Backfilling (Eembedded × 0.5) Backfilling (Eembedded × 1.5)
0.635 0.594 0.723
458.11 436.78 500.78
0.918 0.896 0.949
4.1.1 Dynamic operation and clock frequency limitation - comparison The GridKaARM setup allows for an additional comparison with an alternative cluster operation mode, based on studies conducted in Ref. [26] regarding the installed hardware’s power consumption. In this study, the CPUs clock frequency was scanned to find the maximum power efficiency. The latter is defined as the ratio of CPU performance to the average power consumption. In this case, CPU performance was quantified using the HS23 score [27]. At the maximum derived in Ref. [26], power consumption was reduced by around 40 %, whilst the HS23 score only dropped by 19 %. If the cluster’s CPUs are permanently limited to the associated frequency, carbon emissions can be reduced similarly to the dynamic operation. To still meet the requirement of a constant compute target, the 19 % drop in the HS23 score was compensated by additional hardware and the accompanying increase in embedded emissions. The results of both operation methods are shown in fig. 4. Besides the nominal simulation, dynamic operation results are again displayed as a function of the power consumption ratio Pidle/Pmax . A comparison of the results shows that limiting the clock frequency at the medium workload leads to increased emissions caused by the additional hardware. The dynamic operation reaches its optimum at the trivial case of u = 1. For the backfilling scenario, however, limiting the clock frequency results in an emissions reduction of almost 20 %, thereby significantly outperforming dynamic operation of the GridKaARM setup regardless of the power consumption ratio. Thus, the feasibility of such a clock frequency limit is, not only highly dependent on the hardware but also affected by the cluster’s typical workloads. However, given a clear power efficiency maximum is found, a clock frequency limit could be combined with dynamic operation of a computing cluster. A simulation of this combination is not part of this study but offers an interesting additional test case.
14
Etotal (uopt. ) Etotal (u=1)
Section 4 Simulation results
1.0
0.9
GridKaARM (backfilling workload) GridKaARM (medium workload) GridKaARM clock freq. limitation (backfilling workload) GridKaARM clock freq. limitation (medium workload)
0.8 0.0
0.2
0.4
0.6
0.8
1.0 Pidle Pmax
Figure 4: Relative emissions compared to constant operation Etotal (uopt. )/Etotal (u=1) as a function of the power consumption ratio Pidle/Pmax for the GridKaARM setup in the medium and the backfilling scenario. The nominal power consumption ratio is marked. The horizontal line indicates the relative emissions, assuming constant utilisation of the cluster at a limited clock frequency.
15
Section 4 Simulation results 4.1.2 Emission operation threshold - validation
In order to verify the emission minimisation results, in particular the derived threshold’s stability, a validation was carried out based on the 2023 and 2025 data. As an example, the BAFmodern setup’s backfilling scenario was used. The extrapolation defined in section 3 was applied. A comparison between the targeted utilisation utarget , associated to the threshold Xemission determined via the 2024 data, and the utilisation uextra. reached with the extra. extrapolated thresholds Xemission is listed in table 6. Resulting deviations are within two percent, which implies that the compute target is almost perfectly met in both validation periods. extra. The extrapolated thresholds Xemission are displayed in fig. 5. In addition both are target compared to the threshold Xemission , individually derived to match utarget in each validation period. This comparison yields relative deviations of 1.42 % and 0.68 % for the years 2023 and 2025, respectively. Therefore, taking into account the changed share of renewable energies, no significant deviations are found within both validation periods. Table 6: Target utilisation utarget derived via the 2024 data and utilisation obtained in a validation period uextra. taking into account the changed share of renewable energies for the BAFmodern setup’s backfilling scenario. Associated emission thresholds are also listed. Validation period
uextra.
extra. Xemission / kgCO2 /MWh
utarget
target / kgCO2 /MWh Xemission
2023 2025
0.648 0.648
476.60 463.55
0.635 0.635
469.94 460.44
4.2 Optimisation with respect to operational costs Due to a lack of data, optimisation with respect to operating costs was solely carried out for the two different BAF computing cluster setups. Both setups were simulated under the assumption that 100 % of the electricity demand is purchased on the spot market. A value of Cdemand = 100 €/kW/year was assumed for the demand charge. This was again only intended to serve as a rough estimate, as grid fees in Germany generally depend on the supplier, the region and the duration of use. The acquisition costs per logical core hour were assumed to lie at Cacq., CPU hour = 5.35 × 10−4 €/h for both scenarios. This value was determined based on the costs of BAF hardware currently in use. The calculation only incorporates purchase prices for CPUs and RAM while additional costs for infrastructure etc. are excluded. An internal comparison between original acquisition costs in early 2017 and the costs for more recent expansions in 2022 and 2023 revealed that the acquisition cost has remained approximately constant to date. It is difficult to predict how acquisition costs will change in the future. However, with global developments in the field of artificial intelligence and the associated demand for RAM modules [28] a cost increase seems increasingly likely. The assumed values used for both setups in terms of power consumption, i.e. Pmax and Pidle , still correspond to those listed in table 1. Similarly, all three workload scenarios from table 3 were also reused.
16
CO2 Epower / kgMWh
Section 4 Simulation results
800 600 400 200
01/01/2023
01/04/2023
CO2 Xemission = 458.11 kgMWh
01/07/2023
01/10/2023
CO2 extra. Xemission = 476.60 kgMWh
01/01/2024 Date CO2 val. Xemission = 469.94 kgMWh
CO2 Epower / kgMWh
(a)
800 600 400 200
01/01/2025
01/04/2025
CO2 Xemission = 458.11 kgMWh
01/07/2025
01/10/2025
CO2 extra. Xemission = 463.55 kgMWh
01/01/2026 Date CO2 val. Xemission = 460.44 kgMWh
(b)
Figure 5: Validation of the calculated threshold for the backfilling scenario in the years (a) 2023 and (b) 2025. Both validation periods display the original threshold value extra. Xemission determined via the 2024 data, the extrapolated threshold Xemission taking into account the changed share of renewable energies as well as the target threshold Xemission derived individually in each validation period to match utarget .
17
Section 5 Conclusion
Cost minimisation results are listed in table 7 for both setups. Similar to the results in section 4.1, fig. 6 additionally illustrates total and individual costs as a function of the utilisation u in case of BAFmodern ’s backfilling scenario. As expected, a decreased utilisation causes not only an increase in Cidle , but also in Cdemand and Cacq. . The latter both represent costs connected to the additional hardware, required to maintain a constant compute target. A comparison of all listed scenarios demonstrates that the greatest cost-saving is achieved from simulating a dynamic operation under heavy cluster utilisation, as was also the case for the carbon emission studies. Nevertheless, the relative savings for all tested scenarios lie below one percent. This limited saving potential can be attributed to the dominating acquisition costs Cacq. , as shown in fig. 6. They rise for utilisations of u < 1, as more hardware is required to keep the compute target constant. However, the actual acquisition costs for different setups’ hardware might vary significantly. To assess the potential of a dynamic operation as a function of the acquisition costs, further simulations were carried out for the BAFmodern ’s backfilling scenario. Results for an acquisition cost reduction of 50 % are shown in fig. 7. While Cacq. is no longer the dominant contribution for u ⪆ 0.7, the relative savings compared to Ctotal (u = 1) are still below 2 %. Consequently, a dynamic operation can, even under the assumption of reduced acquisition costs, only achieve quite limited savings. Table 7: Optimisation results to minimise operational costs for both BAF setups and workload scenarios. Listed is the optimal utilisation uopt. , the associated threshold Xcost beyond which a computing cluster switches to idle state, and the relative cost compared to constant operation Ctotal (uopt. )/Ctotal (u=1). In case that a constant operation yields the lowest total costs, no threshold on Cpower is specified. Workload scenario
uopt.
Xcost / €/MWh
Ctotal (uopt. )/C
total (u=1)
BAFdefault Medium Heavy Backfilling
0.999 0.996 0.985
470.78 351.70 200.14
0.999 0.998 0.992
BAFmodern Medium Heavy Backfilling
0.998 0.996 0.983
438.30 330.05 189.32
0.999 0.998 0.991
5 Conclusion Germany’s growth in renewable energies is inevitably linked to relatively large fluctuations with respect to the total energy production. Large power consumers whose operation can be dynamically controlled are able to partially compensate these fluctuations. Through trading on the electricity spot market such a dynamic operation offers potential savings in carbon emission and operating costs. This study investigated the feasibility of dynamically
C / 104 e
Ctotal Cdemand uopt. = 0.983
5 4
Coperation
Cacq.
Cidle
ncores
2.5 2.0
3
1.5
2
1.0
1
0.5
0
0.2
0.4
ncores /104
18
Section 5 Conclusion
0.6
0.0
0.8 u
5 Ctotal Cdemand uopt. = 0.954
4
Coperation
Cacq.
Cidle
ncores
2.5 2.0
3
1.5
2
1.0
1 0
ncores /104
C / 104 e
Figure 6: Display of total costs alongside the individual contributions (left y-axis) and number of logical cores needed to maintain the default compute target (right y-axis) as a function of the utilisation u. The optimal cluster utilisation is marked by the black vertical line.
0.5 0.2
0.4
0.6
0.0
0.8 u
Figure 7: Display of total costs alongside the individual contributions (left y-axis) and number of logical cores needed to maintain the default compute target (right y-axis) as a function of the utilisation u. The optimal cluster utilisation is marked by the black vertical line.
Section 5 Conclusion
19
operating a computing cluster used in science. These computing clusters represent an ideal test case, since short-term delays caused by any dynamic operation are often negligible as long as the overall compute target remains constant over prolonged periods. On basis of the German energy generation data of 2024, dynamic operation for five different computing clusters installed at research institutes was simulated. The results were compared with respect to a potential carbon emission and operational cost reduction. Savings with respect to the operating costs during the simulation period were found to be quite limited due to the dominant acquisition costs. For all tested setups and workloads relative savings stayed below 1 % with respect to a standard non-dynamic operation. In contrast, the results regarding carbon emissions showed more pronounced differences between the various cluster configurations. The decisive factor for the effective implementation of dynamic operation was found to be the relative idle power consumption Pidle/Pmax . Setups with a comparatively high value allowed for little to no emission reduction, whereas the BAFmodern setup with Pidle/Pmax = 0.15 achieved savings of up to 8 %. To validate the results, dynamic operation of the BAFmodern setup’s backfilling scenario was additionally simulated for the years 2023 and 2025. Taking into account the changed share of renewable energies, a deviation from the targeted relative utilization of no more than 2 % was attainable. This demonstrated, at least within the limited validation period, that a fixed annual compute target can be met consistently. Apart from the two specific setups, simulations were also carried out for variable modelling parameters. Values for relative idle power consumption and acquisition costs were varied independently in order to assess the effects of potential future developments or different hardware setups. To at least partially evaluate the effect of the embedded emissions’ large uncertainties additional testes were carried out. It was demonstrated that the dynamic operation allowed for an emission reduction within the chosen uncertainty band. The GridKaARM setup enabled a comparison between dynamic operation and continuous operation at a limited clock frequency. Results showed to be quite hardware- and workloaddependent but combining both methods might be a viable option to increase the emission reduction in the future. While this study is intended to serve as a proof of concept, demonstrating the potential for savings, particularly in terms of carbon emissions, further steps are necessary before implementing the dynamic operation of a cluster in practice. Among other simplifications used in the simulation, an interface to stop and restart the cluster operation is mandatory. Computing jobs must therefore be able to be paused and resumed flexibly. Currently it is assumed, that switching between idle and normal operation happens instantaneously and can be performed as often as required. The number of these switching operation increases for lower utilisation values. In case of the BAFmodern setup’s backfilling scenario the cluster had to pause and resume operation roughly 230 times in the simulated period. For practical application, however, a ramp-up and ramp-down time must definitely be taken into account. This might affect and shift the optimisation results. In addition, a more sophisticated scaling factor has to be implemented and replace the simple scaling via the number of logical cores, currently utilized in this study. Otherwise, a realistic consumption, especially for inhomogeneously structured computing clusters consisting of various CPU and accelerator modules is hard to estimate.
Section References
20
Acknowledgements The present analysis would not have been possible without the publicly available energy production and spot market price data by the Fraunhofer Institute for Solar Energy Systems (ISE). The authors would thus like to express their gratitude and support for open data. In addition, the authors are grateful for access to the resources of the BAF2 cluster at the University of Bonn, the GridKa cluster at the Karlsruhe Institute of Technology, as well as the DEEP-System of the Jülich Supercoumpting Centre to simulate the different setups. This work was supported by the Federal Ministry of Research, Technology and Space (BMFTR) within the project FIDIUM (Förderkennzeichen 05H21PDRC1) and SUSFECIT (Förderkennzeichen 05D25PD4). The collaboration within the PUNCH4NFDI consortium in the NFDI (Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) — project number 460248186) was instrumental in carrying out the project. It was also supported by the environment of the cluster of excellence EXC 3107 “Color meets Flavor”. We are very grateful for the support by all participants and collaborators of the SUSFECIT consortium, most importantly Markus Schumacher and Michael Böhler.
References 1.
Arrhenius S. On the influence of carbonic acid in the air upon the temperature of the ground. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 1896; 41:237–76. doi: https://doi.org/10.1080/14786449608620846
2.
IPCC. Climate Change 2023: Synthesis Report. Contribution of Working Groups I, II and III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Geneva, Switzerland: IPCC, 2023 :35–115. doi: https://dx.doi.org/10.59327/IPCC/AR6-9789291691647
3.
Stöver B, Reuschel S, Wolter MI, Daßler J, and Bernardt F. Die Kosten des Klimawandels für Deutschland 2025-2050. GWS Research Report Series 25-1. GWS - Institute of Economic Structures Research, 2025
4.
United Nations Framework Convention on Climate Change. Paris Agreement: Preamble. 2015 Dec
5.
Federal Ministry for Economic Affairs and Energy. Renewable Energy Sources in Figures. National and International Development, 2016. Federal Ministry for Economic Affairs and Energy (BMWi), 2017
6.
Brown T, Schlachtberger D, Kies A, Schramm S, and Greiner M. Synergies of sector coupling and transmission reinforcement in a cost-optimised, highly renewable European energy system. Energy 2018; 160:720–39. doi: https://doi.org/10.1016/j.energy.2018.06.222
7.
Federal Ministry for Economic Affairs and Energy. Status and Development of the German Data Centre Landscape. Executive Summary. Federal Ministry for Economic Affairs and Energy (BMWi), 2025
8.
Cao Z, Zhou X, Hu H, Wang Z, and Wen Y. Toward a Systematic Survey for Carbon Neutral Data Centers. IEEE Communications Surveys & Tutorials 2022; 24:895–936. doi: https://doi.org/10. 1109/COMST.2022.3161275
9.
Bundesnetzagentur. Record High for Solar Generation in Each Quarter. The electricity market in 2025. 2026 Jan. Available from: https://www.smard.de/page/en/topic- article/217400/ 219038/record-high-for-solar-generation-in-each-quarter [Accessed on: 2026 Feb 19]
10.
Freyermuth O, Wienemann P, Bechtle P, and Desch K. Operating an HPC/HTC cluster with fully containerized jobs using HTCondor, Singularity, CephFS and CVMFS. Computing and Software for Big Science 2021; 5:9. doi: https://doi.org/10.1007/s41781-020-00050-y
11.
Suarez E, Amaya J, Frank M, Freyermuth O, Girone M, Kostrzewa B, et al. Energy efficiency trends in HPC: what high-energy and astrophysicists need to know. Frontiers in Physics 2025 Apr; 13. doi: https://doi.org/10.3389/fphy.2025.1542474
Section References
21
12.
Lang VS, Bhalla NK, Gurdasani SS, and Niknejadi P. Know your footprint—Evaluation of the professional carbon footprint for individual researchers in high energy physics and related fields. npj Climate Action 2025; 4:28. doi: https://doi.org/10.1038/s44168-025-00232-7
13.
Suarez E, Amaya J, Frank M, Freyermuth O, Girone M, Kostrzewa B, et al. Energy-aware operation of HPC systems in Germany. Frontiers in High Performance Computing 2025; 3. doi: https : //doi.org/10.3389/fhpcp.2025.1520207
14.
Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Scientific data 2016; 3:1–9. doi: https://doi.org/10.1038/sdata.2016.18
15.
Bruers B, Cruces M, Demleitner M, Duckeck G, Düren M, Eich N, et al. Resource-aware Research on Universe and Matter: Call-to-Action in Digital Transformation. The European Physical Journal Special Topics 2024. doi: https://doi.org/10.1140/epjs/s11734-024-01436-4
16.
Gesetz zur Steigerung der Energieeffizienz in Deutschland (Energieeffizienzgesetz). Bundesrepublik Deutschland, 2023
17.
Gandhi L. Energy-Charts.info: Public net electricity generation in Germany. 2026. Available from: https://www.energy-charts.info/charts/power/chart.htm?l [Accessed on: 2026 Feb 19]
18.
Gandhi L. Energy-Charts.info: Carbon dioxide emissions from electricity generation in Germany. 2026. Available from: https://www.energy-charts.info/charts/co2_emissions/chart.htm?l [Accessed on: 2026 Feb 19]
19.
Gandhi L. Energy-Charts.info: Annual renewable share of public net electricity generation and load in Germany. 2026. Available from: https://www.energy-charts.info/charts/renewable_share/ chart.htm?l [Accessed on: 2026 Feb 19]
20.
Eicker N, Lippert T, Moschny T, and Suarez E. The DEEP Project An alternative approach to heterogeneous cluster-computing in the many-core era. Concurrency and computation: Practice and Experience 2016; 28:2394–411. doi: https://doi.org/10.1002/cpe.3562
21.
Jülich Supercomputing Centre (JSC). 2026. Available from: https://www.fz-juelich.de/en/jsc [Accessed on: 2026 Apr 16]
22.
Grid-Computing-Zentrum Karlsruhe - GridKa. 2026. Available from: https://www.scc.kit.edu/ forschung/gridka.php [Accessed on: 2026 Apr 16]
23.
thinkstep AG and Dell Technologies. Life Cycle Assessment of Dell PowerEdge R740. Technical Report. Prepared by thinkstep AG on behalf of Dell Technologies. Dell Technologies, 2019 Jun
24.
Tannu S and Nair PJ. The Dirty Secret of SSDs: Embodied Carbon. SIGENERGY Energy Inform. Rev. 2023 Oct; 3:4–9. doi: https://doi.org/10.1145/3630614.3630616
25.
Shmueli E and Feitelson DG. Backfilling with lookahead to optimize the packing of parallel jobs. Journal of Parallel and Distributed Computing 2005; 65:1090–107. doi: https://doi.org/10.1016/ j.jpdc.2005.05.003
26.
Krull AA, Schnepf MJ, Fischer M, and Petzold A. Experience with ARM WNs at the WLCG Tier1 GridKa. EPJ Web Conf. 2025; 337:01147. doi: https://doi.org/10.1051/epjconf/202533701147
27.
Szczepanek N, Britton D, Girolamo AD, Ketele E, Glushkov I, Giordano D, et al. HEP Benchmark Suite: Enhancing Efficiency and Sustainability in Worldwide LHC Computing Infrastructures. 2024. doi: https://doi.org/10.48550/arXiv.2408.12445
28.
Zhang T, Mailthody VS, Sun F, Ma L, Newburn CJ, Zhang T, et al. From Minutes to Seconds: Redefining the Five-Minute Rule for AI-Era Memory Hierarchies. 2025. doi: https://doi.org/10. 48550/arXiv.2511.03944. arXiv: 2511.03944 [cs.AR]